Better Typing

The main changes since the 0.43.0 release are the addition of typeclasses, basic value checking, and explicit function type parameters.

Typeclasses cannot yet be used as explicit constraints in function signatures, so their value in modeling data is limited. But at least we can now have one add function name for both integers and doubles. Also, packing and unpacking has been re-implemented using a new Packable type class. This means we no longer need the special pack and unpack descriptors in signatures.

Value checking is important since morloc can define multiple definitions for one term. For example, it is legal to write:

x = 1
x = 2

This would not redefine x, as is done in many languages, but would rather associate both values with the variable name and attempt to disambiguate them later (which in the past implementation would have arbitrarily picked the last one). Now I have a very rudimentary value checker that will check for contradictions between primitives. It cannot descend past a source function call. In the future, I will need to extend the value checker to compare different sourced functions. This will likely have an LLM solution.

Explicit function parameters are now added to function signatures to provide an order for the generic type variables. For example:

snd a b :: (a, b) -> b

This deviates from Haskell syntax, but clarifies the relationship between the morloc type signature and type signatures in other languages, such as the C++ prototype:

template <class A, class, B>
B snd(tuple<A,B>);

This also allows us to conveniently refer to functions as parameterized types, e.g.: snd Int Bool. Possibly such type functions could be used in signatures as well. I will explore this later.

Nearly Useful

The full influenza case study, a re-implementation of OctoFlu, is supported in this release. This proves that morloc can be used to solve non-trivial problems. Here are the main advancements of this release:

1. Infer all concrete types directly from the general type
2. Allow file inputs rather than only raw JSON -- this allows large data sets to be processed without hitting argument size limits
3. Better (though still far from good) debugging options and error messages
4. Clean import/export system with wildcards
5. Support for eta-reduction
6. Many bug fixes and greatly extended test coverage

However, the language is still weak in many areas:

1. No type classes
2. No effect handling (e.g., exceptions, mutations, non-determinacy)
3. Weak record/table/object support
4. No pattern matching or sum types
5. No binary operator support (I'm getting a little tired of writing add rather than +)
6. Limited debugging features
7. Limited language support
8. Slow compile times (due to one specific issue in the frontend type system)
9. Inefficient serialization scheme (uses JSON currently, should convert to some sort of remote procedure call system)
10. No formal specification of the type-system -- the conversions from general to concrete to serial, the resolution of ambiguous trees, the propagation of types through segmentation, the threading of arguments -- all this is very involved but not yet mathematically defined. I am not confident that it is all sound.
11. No shiny paladin salesperson, only a grumpy morlock who thinks only about problems

Where scoping

This release adds scoped where to morloc and fixes several subtle bugs in the typesystem.

Under the hood, the entire architecture has been refactored. Previously concrete and abstract typechecking were done in a single step, now first the general types are inferred, then the trees are disambiguated, and finally the concrete types are inferred. Also, typechecking is now done AFTER the raw expressions have been parsed and desugared into the set of ASTs that will be exported from a module. There are many, many more changes in the implementation. If you are curious, read the commit messages.

Pretty good typechecking and serialization

This release sets the foundation for morloc. Basic typechecking/inference, code generation, interoperability, and serialization are all working well. Finally morloc is sufficiently developed to be useful.

The main future goals break down as follows:

• Richer type system - typeclasses, "shapes", semantic types (probably use a logic engine like z3)
• Effect handling and error/warning propagation
• Optimization - all current optimizations steps are basically stubs
• Doxygen-like documentation, caching, manifold hooks and such (see the last release)
• Improved build system
• Support for many more languages and a streamlined language onboarding process
• The MorlocIO package manager and community portal
• MorlocStudio

PyCon 2019

This release marks the version of morloc that was used in the poster presented at PyCon 2019 in Cleveland.

Minimal Haskell Prototype

This release presents a very simple Morloc prototype. It is mostly experimental and will change greatly in the future with no attempt to preserve backwards compatibility.

This prototype includes

• A simple, typed, functional scripting language
• A compiler to translate these scripts into RDF graphs and then executable code
• Simple type checking
• Support for Python and R
• A system for specifying language-specific types and transforming the data as needed
• Syntax for specifying type constraints

Pre-release of Haskell prototype

This prototype is (currently) much less sophisticated than the C prototype. However, the code is far more elegant and will serve as a more flexible foundation for future development.

It can currently run R code in a simple shell interface. For example:

> sum [1,2,3]
6

This passes the Morloc vector [1,2,3] into the R function sum and returns the result.

This pre-release is an experimental foundation for the Morloc language. The syntax and features will change wildly in the future with no attempt at maintaining backwards compatibility.

Final version of the C prototype

This is the final version of the C prototype.

The features are described in the README. Here is an overview:

• integrated R, Python, and Bash through a simple type system

• workflows are pull-based graphs

• explores the "manifold" template idea and multi-dimensional workflows

• compilation exposes all exported functions through the manifold nexus

• allows checks and effects to be added outside of the core workflow

This prototype will not be maintained in the future.