Rec Data Processing Framework
Rec using a updated version Rhino as JavaScript engine, it support several ES6 features to help you for better development experience.
Standadrd interfaces is defined in following files:
Rec has several core concepts to represent data and related stuff, core idea
is represent each data entry as a Record.
interface Record {
get(value: string): string;
keys(): string[];
}
Basically a record is just a string-indexed-value object, which has specific schema, i.e. the accessor expression, to get the values.
in JavaScript, the records can be accessed as a JavaScript Object, like if
you have a Record contains key name and address, you can just using let
destructive assignment:
let {name, address} = <the record>;
Also a record have the keys method to get all it’s accessible keys, and
this can be easily to encode a record using JSON.stringify.
Other important concepts are the processing components, in Rec, there are
3 different components to support different processing scenarios: the Source,
to obtain data; the Target, to save/transfer data; and Tee, to add processor
in middle.
Source is the component to obtain data, e.g. a CSV file, or a server socket.
A source can be generated from a Rec method, or generated by a existing source object. Once it is generated, the existing source is considered as parent of the new one, and new one considered as child of existing one.
It is important to understand that a source is not reentrant, once you direct a source to a target, neither it’s child source or parent source is no longer valid. And this invalid status is related, meaning, if it’s child is invalid, the parent should also be invalid, or vice versa.
Each source have 3 different methods:
Source#to(target: Target)
Direct the source to a specific target, either it is a blocking process to put all data to the target, or it is a non-blocking process to reactively call the target when the source got data.
Source#tee(tee: Tee): Source
Attach a tee to a source, this will create a new source, which is lazy / reactive based on the implementation of source.
Source#filter(predicate: WrappedPredicate<Record>): Source
Attach a filter to a source, this will using a predicate, as this is a plain Java
method, so you need call Rec#pred method first to wrap a JavaScript predicate
function.
This method will create a new source.
A target represents the destination of data, e.g. a consumer function, or a database connection (to save the data).
A target can be reused many time, and can attach a tee.
Target#tee(tee: Tee): Target
Target#tee will create a new Target object, but the previous object is still available
to use.
Tee is to represent the process in-middle of data aquiring and saving, like a T joint pipe (and that’s the origin of it’s name), e.g., saving the intermediate result, or do some aggregating/validating process.
A tee can be easily convert to a source if it contains aggregated data.