Log 120216

The interpreter can do symbolic evaluation but it can't add two numbers yet. Reason, it has a pluggable architecture for combinators and I also need to implement the simple operator overloading scheme.

This is a big no-no for compiler writers, of course, but I can say I care much. But still, here's a definition for dyadic minus.

class IntegerMinus: public VMObjectCombinator {
public:
    IntegerMinus(VM* m): 
           VMObjectCombinator(VM_OBJECT_FLAG_INTERNAL, m, 
                                    "System", "intminus") {
    }

    IntegerMinus(const IntegerMinus& d)
        : IntegerMinus(d.machine()) {
    }

    VMObjectPtr clone() const {
        return VMObjectPtr(new IntegerMinus(*this));
    }

    VMObjectPtr reduce(const VMObjectPtr& thunk) const override {
        auto tt  = VM_OBJECT_ARRAY_VALUE(thunk);
        auto rt  = tt[0];
        auto rti = tt[1];
        auto k   = tt[2];

        VMObjectPtr r;
        if (tt.size() > 6) {
            auto arg0 = tt[5];
            auto arg1 = tt[6];

            auto i0 = VM_OBJECT_INTEGER_VALUE(arg0);
            auto i1 = VM_OBJECT_INTEGER_VALUE(arg1);
            auto ir = i0 + i1;
            r  = VMObjectInteger(ir).clone();
        } else {
            VMObjectPtrs rr;
            for (uint i = 4; i<tt.size(); i++) {
                rr.push_back(tt[i]);
            }
            r = VMObjectArray(rr).clone();
        }

        auto index = VM_OBJECT_INTEGER_VALUE(rti);
        auto rta   = VM_OBJECT_ARRAY_CAST(rt);
        rta->set(index, r);

        return k;
    }
};

At the moment I don't feel like copying this code for every operator, can't solve it reasonably with a macro or template (or I don't know how), and don't want to solve it with a general class and a function pointer. I'll wait a day, something will come up.

Much Weirdity

After more debugging it seems to evaluate fine. Weird stuff you can do too, shown below.

namespace Test (

    def f = [ X, Y -> X ]

    def g = [ f X -> f X X ]

)

using Test

def main = g (f 1)

Which gives

[marco@stallion src]$ ./egel ../tests/match.eg 
1

Okay, for those who assume this is a feature, this is fully intentional. Egel has eager combinator rewriting semantics so when evaluating g (f 1) it will try to reduce f 1, which fails since f is a dyadic function, therefor remains constant. After that g matches on f 1 and f 1 1 is reduced to 1. Completely according to semantics.

I hope to lift that behavior such that I can implement Mathematica style symbolic manipulation, though that has a far more complex operational semantics. But who knows?

Hello 42!

Well. I passed a major milestone tonight. The egel interpreter evaluated a hello world application.

Given input

def main = 42

It produces bytecode

SYMBOLS: 
       0:main
DATA: 
       0:42
       1:main (4)
begin
000000 takex r1 r5 r0 i0
0x0009 fail 0x001f
0x000e data r6 0   ; 42
0x0015 set r1 r2 r6
0x001c return r3
0x001f array r7 r2 r1
0x0026 concatx r8 r7 r0 i4
0x002f set r1 r2 r8
0x0036 return r3
end

Which when run

[marco@stallion src]$ ./egel 42.eg 
42

It core dumps on more complex expressions, of course. I hand checked generated code, which looks okay-ish for small examples, so I'll be stuck debugging the bytecode interpreter for a while.



Guess I'll commit to github after that.

Log 301116

Well, after a long time of neglecting the sources on my computer I somehow got in the mood to code again. Somehow I needed to get rid of all the wild ideas I wanted to implement. So, I've been coding on-and-off for some time now.

Stuff I did.

• Simplified the syntax even further.
• Combinator lift routines.
• Rewrote the module loader such that it can handle interactive mode.
• Got rid of methods in favor of 'overloaded' operators.
• Started on a bytecode interpreter.
• Registered a github page.

It now generates very faulty bytecode. But I am happy that at least I got to the point that C++ accepts it.

I rushed the bytecode generator too much. This needs some re-thinking how to maintain several invariants when traversing the AST. And that for a trivial language which will never be used much. Ah well.

Progress

So, I finally solved a major part of the puzzle. Now the rest.

Change of Plans

I decided to go into a different direction.

Apache Data Flow Proposal

Dataflow is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow (a cloud service). Dataflow also brings DSL in different languages, allowing users to easily implement their data integration processes.

Yeah, you can do it this way. Inevitably, people are moving away from map-reduce and one of the best manners of designing around more functionality is mapping stream processing functions onto cluster hardware.

That sounds simple enough for a language, but isn't. The two major hurdles are: For one, you're dealing with mobile code - and only languages like Java and Python seem to provide that (and, of course, Erlang.) Second, you're dealing with mobile data which means you're going to fall back to a form of explicit memory management since it's very costly to move around large objects.

Mobile code, mobile data. It's all solvable given current techniques but all-in-all it makes the most sense to target an impure imperative OO language with support for mobile code and combinator based programming. Preferable with primitives to turn an abstract data type into a service, or the reverse, when needed to prevent major refactoring and seamless scalability.

So, I am stuck designing a language for this since a Java library with a Scala interactive front-end will probably be so much better.

I should get back to my language, but it ain't easy. Mobile code isn't trivial, mobile data and seamless scalability is even further away.