Javascipt 2011

A lot has happened since my last post:
- node.js
- javascript became mainstream :-)
- html5 with websockets
...

I did quite some javascript programming in the last two years,
and start blogging again :-)

Nihilogic

Nihilogic shows it again:
Diggattack


Making fast games in javascript is definitely possible!

Antisocial

Well this is nice demo!

from the javascript demo scene: Antisocial

Code and design was done by Gasman

Reduce the number of inner function creations and closures

Oft it is easy to create an anonymous function to handle an event or perfrom a little task:

Look at this example:

var b = function (/*array*/a ) {
     a.forEach(function(_){ if( _=_.text) alert(_);});
}

The function which perfroms the alert is in javascript an object and created once for the whole array but it is still created each time when b is called.

Functions are objects with a state fromed by a closure.
In this case the closure is not needed, as the function does not use any variable or parameter of b.
The new creation of this function on each call of x is thus just a waste of time and has to be cleaned up by garbage collection.

Wy not create the function once and store the function and reuse it lateron. But where?!

We could use the function object which we store in b.
To reference it we give it a name F
and our anonymous alert function is stored in F.alert_text:

var b = function F () {
    a.forEach( F.alert_text = F. alert_text || function(_){ if( _=_.text) alert(_);});
}

It looks almost the same, but this time for each is perfomed with F.alert_text which is set to it's previous value or set to the anonymous function if not yet defined.

When we have a little global update function to extend (X) an object with properties of an other one like this:

var X = function(/*target*/t,/*source*/s){ for ( var p in s ) {t[p]=s[p];} return t; };

we can rewrite the above code to:

var b = X( function F () {
    a.forEach( F.alert_text );
},{
    alert_text : function(_){ if( _=_.text) alert(_);}
});

This time F.alert_text is even not tested and set to the same vaue, each time when b is called.

Note: this tip is only usefull if the extracted function (here alert_text) is really independent of the state defined by the function calling it. ( here b).

Anonymus temporary variables: _,__

A post on ajaxian about this blog entry by Michal Till and a especially a comment by Andrea Giammarch delivered the idea for this blog entry:

When we write code we often do repeat in our code property lookups over and over

Example:

 
if( node && node.nextSibling && node.nextSibling.className=='x' ) return node.nextSibling.className;

We can use an extra variable to make the code much sorter, cleaner and even faster:
Simply declare some anonymous temporary variables at the beginning of your function

 
function(..) { var _,__; 

Use them where ever you like:

 
if (_=node) if(_=_.nextSibling) if((_=_.className)=='x') return _;

_=a;a=b;b=_ // swap 

Why '_' ? Well because it is almost not visible. It's a perfect name for a temporary placeholder, the name is just not relevant, the meaning is revealed by it's immediate context.

Look at the next loop over an array to perfrom an alert for each element with a text property:

 
for( var i=0, l=a.length; i<l;i++)  if (a[i].text) alert(a[i].text); 

We could use _ like this:

 
for( var i=0,l=a.length;_=a[i],i<l;i++)  if(_=_.text) alert(_);

Or in a more extreme way:

 
for(__=a,___=0,____=__.length;_=__[___],___<____;___++)  if( _=_.text) alert(_);

While this is a lot longer, when we remove the punctation it is just for a 0 length if text alert and thus emphasizes the important elements.

Whereas the version with 'i,l' was for var i 0 l a length i l i if a i text alert a i text and our brain has to filter it.

Of course as an experienced programmer you are used to the i,l,i++ etc. and you will recognize it as a typical loop pattern,

But with the _ vars we express it more explicit: don't care about that looping stuff. You can copy it and replace 'a' and it will work. We could get used to this pattern as well and it may be a help for beginners.

Or we use the new forEach:

 
a.forEach(function(_){ if( _=_.text) alert(_);});

Oh this creates a new function each time the loop is used, but that's a topic for the next blog entry ;-)

Extending Javascript the easy way!!

Ometa by Alessandro Warth is a great environment, a must see!

Think of extending javascript with some little language extensions?
Writing a new / extending an existing compiler gives you a headache?
With OMeta/JS you can do it right in your javascript code:

Extend the Ometa Javascript parser (JSParser implemented in javascript) with a lets say say statement: Here it just does an alert();
Then switch the translator an execute the your new Javascript language right there:

eval(readFile('Compiled_JavaScript_Compiler'))

ometa EJSParser <: JSParser {
  isKeyword :x = ?(x == 'say') | super(`isKeyword, x),
  stmt         = "say" expr:x sc -> ["call", ["get", "alert"], x]
               | super("stmt")
}
EJSParser.matchAll("if (true) say 1+1", "stmt")

oldTranslateCode = translateCode

translateCode = function(s) {
  var tree = EJSParser.matchAll(s, "topLevel")
  var code = JSTranslator.match(tree, "trans")
  return code
}

// evaluate the next statement in a different "do-it" than the above
say "hello world"

translateCode = oldTranslateCode

Some said Lively Kernel uses it. Well svg is nice. But why not stay with DOM nodes and be portable...

Parsing Javascript, Parsing with Javascript 2

Javascript parser/interpreter in javascript: Narcissus

Some other stuff on parsers in javascript: Parser Combinations

An own parser language in javascript OMeta

StopIteration and undefined

Good to know:

StopIteration and undefined are mutable globals by Dave Herman: "Luckily, there's generally some reliable way to generate such a value without having to hope that no one has reassigned a global variable."

Declaring Object Structures

There are different ways of creating object structure.
One is declaring a structure:

Declerative programming emphasises on the principle of concentrating on WHAT is the result and not HOW it that result achieved.

SQL is for example a declerative query language. As a consequence there are lots of implementation variations and they can be improved without the changing the user code.

There is a good language for everything, look how we can declare object structures for example a HTML DOM structure:

• Computational Construction
  
  This involves calling an API, and create instances directly with function calls. It is thus not delerative at all:
  
  

  
  var body = document.getElementByTag('body')[0];
    var div = document.createElement('div'); body.appendChild(div); div.setAttribute("class","c1 c2");
       var p = document.createElement('p'); div.appendChild(p);
         var bla = document.createTextNode("bla bla"); p.appendChild(bla);
  

  
  
  This a very verbose way to build object structures.
  We are using functions to declare a structure by 'computing' it, altough there is not much computing going on, more wiring structures together over and over again.
  Each each library of functions creates a sort of new 'language'. But as it are functions the emphasis is still on computation. The disadvantage is that there is so much freedom in ordering and verbose calling going on, that we can not capture the relevant information at once: What is really going on? Compare this to the next code resulting in the same structure:
  
  

  
  var bla = document.createTextNode("bla bla");
  var body = document.getElementByTag('body')[0], div = document.createElement('div');
  var p = document.createElement('p'); p.appendChild(bla);
  body.appendChild(div);div.appendChild(p);
  div.setAttribute("class","c1 c2");
  

  
  The focus on HOW we build it, hides the knowledge about WHAT we build.
  



• Representive Data Declaration
  
  In this case we define the final oject structure, what we want, by using representive data structure:
  
  

  
  var html = [{ body : [{}, {div:[{ class: "c1 c2" }, {p: [{},"bla bla"]} ]} ]}
  }]
  

  
  
  We then need extra functional code wich translates the object structure into the final object structure: funtion get_node(description,document){ .... return 'the node';}, which can hide all sort of implementation details.
  
  With this pure data based approach, we are forced to use the elements and semantics provided by the base language. Here we use for eample javascript object literals to express html literals. As javascript has other semantics then html mark up it is not so good at it. It makes it more verbose. See how we have to express that importance of ordering of html child nodes by using arrays, and how we allow multiple object with the same name by using objects with a single attribute. That is why we get these repeating ':[{' ']}' and kind of brackets.
  
  A small advantage is that the parser checks some of the brackets, but as the correctnes depends on a certain combination, the correct 'html' syntax can be checked only at runtime.
  
  Another advantage is that you only need to learn javascript to be able to 'read' html, and not an extra language. On the other side to 'understand' it, you still have to learn the html concepts. And the verbose non suitable notation is not very helpfull.
  
  Ext uses this approach combined with the computational construction.
  
  Other examples of this approach and its limits can be seen when using XML based template languages or even XSLT. Very often XSLT contains computational elements like 'if' 'then' 'else'.
  Expressing this in a declerative language that was not designed to be computational becomes very verbose and besides it then is a sort of 'XML' it does not provide any advantage. We use a lanaguage for what it was not designed, and therefore is not good at. The interpunctation is not optimized. For example XML only knows nested brackets, we can not express directly an expression with 3 elements 'if C then A else B' directly, but have to encode it: 'AB' or simular. To make it simple we then put functional code like javascript in the string attributes or text nodes.
  That is mostly often exactly what template languages do.
  



• Decleration with Function Language
  
  Here we use function calls with objects as parameters in a mixed way, as builders to create the object structure:
  

  
  var body_node = body()
    .div({class:"c1 c2"})
      .p("bla bla").end() 
    .end()
  .end().toString(document);
  

  
  
  This allows a short flexible notation mixing calls and parameter objects.
  
  Note how this approach goes step further in using functions as interpunctation elements. The meaning changes depending on the context. Here a own block/bracket structuring language is developed by using javascript function names.
  Because we are allowed to introduce new function names with n paramters we can change the set of available language elements.
  
  The syntactical correctnes is not controlled by the language parser, but could be at runtime. Throwing a missing end() call exception for example.
  
  This is thus creating a new language using a an other language. We thus can use the programmatic API calls mixed with object strcuters as function parameters as an implementation of the function language deceleration.
  
  jQuery also uses this approach.
  
  





• Dedicated Language
  
  
  This seams to be the shortest and clearest way to declare a structure. Use a dedicated optimzed language, suited to the purpose.
  
  

  
  var body = '<body><div class="c1 c2"><p>bla bla</p></div></body>';
  

  
  
  Note how the function language decleration approach may serve as an implementationof a parser/compiler/runtime system for such a language.
  
  In a first attemp, a pre processor could translate the above code in a appropiate function calls:
  
  

  
  var body_node = body().div({class:"c1 c2"}).p("bla bla").end().end().end().toString(document);
  

  
  
  While many people resist when developers want to define a new language, because things become 'complicated' then this is just part of the truth. Above examples show that without a dedicated language, things do not get clearer. While not learning something 'new', that standard code is very 'hard' to understand so what costs more? The problem is that it is not easy to design a good language and often you would need to extend it, adapt it to changes. You must spreak a fashionable language, you need to find enough developers. You need good debugging capabilities etc. This makes it as hard just like the framework programming where you start depending completly on the framework/compiler developer.
  
  A big part of the problem is that the dedicated language gives you the feeling of being locked in the 'every thing must be expressed in this language/framework' trap. And things start to become even more problematic if in an attempt to 'escape' these limits complete external languages are mixed in with there own different semantics but then need to communicate with each other using API calls, then all sort of adapters libraries, translations are needed and then at the interfaces all sorts of conversion errors and mismatches occur. You are in a trap.
  Just see what happend to the HTML Javascript, java, flash mixing.
  
  A requirement to make a language work is to be able really to hide the implementation for 95% of the cases to developers and allow a simple debugging. This 'all or nothing' requirement is what makes a dedicated language not suitable for single projects. There are how ever some things that can done to embed a dedicated language, and not suprisingly they are the same as for developing a library:
  
  
  
  
  • Take a primary full supported language with debugger libraries as starting point. And embed your stuff in it. Not the other way round.
    First design a good library of you add ons.
    
  
  
  
  • Make the use of the language extension optional. The old framework idea don't call us, we call you approach is causing the locked in the trap. It makes you reacting like a fool, never knowing when you are getting called and why. Debugging is a pain. The developer of the user code must be in control when he calls what. Most developers even SQL programmers simply think in HOW things are done and must be able to programm, test and debug that way.
    
  
  
  
  • Keep the language small and simple. If you can not hide the inner working 100% make them obvious, provide source and allow debugging. C++ startet as a simple preprocessor. The resulting C code could still be debugged and was understandable. Lateron when it became popular debuggers/compilers made it an own language.
    
  
  
  
  • If you are not able to put it a good concise library then you are not able to make it a good language.
    
  
  
  
  
  
  
  

Rendering, HTML/XML to JSON, JSON to HTML/XML

Why? you could ask. Both XML and JSON have strengths and different origins. XML unstructured texts with some markup, and JSON from describing javascript data structures.

HTML is our 'Rendering Engine Language'
Is it not HTML as mixture of incorrect XML with javascript hooks, CSS linkage the main javascript GUI 'rendering' engine?

Well assume we simply wnat to describe HTML within javascript. Why learn different languages HTML/CSS/Javascript if one would be good enough? Using just javascript could remove the artificial break between the active javascript and the declerative XML.

Well there has been some mapping posts:

• Converting between XML and JSONby Stefan Goessner
  
• xml2json.js by Thomas Frank who say you should better use
  
• terracoderby Sam Tsvilik

Stefan Goesner brought up some good points:

• JSON has and ordered list of values: arrays, objects as name/value pairs, with unique names. Each value can be an object an array or a literal string, number etc.


• XML elements have a single set of non named attributes with string values, and has an ordered list of childnodes, wich can be text nodes, and other named child elements.
  

While the set of attributes could easily be directly mapped to some name/value pairs in JSON, this is not the case with the child nodes. They have an order and a name.
The correct way woud thus be an array to preserve order and within the array we then could put text or object with a name value pair:

<p class="c1 c2 c3" style="border: 3px coral solid;font-size:1.5em">
  bla bla
  <ul><li>one</li><li>two</li></ul>
  bla bla
  <input type="button" onclick="alert('click');"/>
</p>

How could we map this to JSON/Javascript like syntax?, we propose using an array where the [0] index contains the attributes, this makes the notation somewhat shorter: Note that XML path uses sets beginning with index 1. So this could maybe justify this decision.

var c1={},c2={},c3={}
json = {p:[{class:[c1,c2,c3],style:{border:[{px:3},'coral','solid'],font:{size:{pt:12}}},
   "bla bla",
   {ul:[,{li:[,"one"]},{li:[,"two"]} ]},
   "bla bla",
   {input:[{type:'button'},{onclick: function(){alert('click');}}]}
]}

Is this much sorter or cleaner then XML itself? I'am not convinced. It still looks ike we misused JSON
But what becomes obvious is that HTML is not that clean as XML. Just look at the class attribute and the style stuff. A language within a language encoded in a string.
I just played a bit around and translated the multi value class attribute to an array, and the style css syntax also to a json like syntax.
Look at the java script code linked to the click event, here it is a real function not just a text.

The advantage could be that HTML/CSS/Eventhandling could all be defined and processed in a single language.

Stefan Goessner and others also wantet to keep accessing elements easy. He even used an non bidrectional XML to JSON mapping to keep access easy. p.ul.li.text could deliver 'one'. While this is a nice idea it is hard to keep it that simple if there is a bidrectional general mapping needed. In our mapping we can access both list items:

p[1].ul[1].li[1] = 'one';
p[1].ul[2].li[1] = 'two';

As always with hierachical structures an processsing whe soon have the desire to
access sets of nodes them using query language like XPATH or JSONP or CSS selectors.
Designers are so used to CSS Selectors jQuery was designed around this.

But back to our language description. In the end its all about language and we just introduced a new 'data' language. More on that in the next post.

Why not using this?

p @ class{ c1, c2, c3} , style{ border{ px:3, coral, solid } , font:size:pt:12 
  { bla bla,
    ul { li:one, li:two }
    bla bla
    button@onclick:alert('click');
  }

A little precompiler could transform this to javascript

JXML	maps to JSON
X1..Xn	"X1..Xn"
X@A1,..,An{E1,..,En}	X:[{{A1,..,{An}},{E1},..,{En}]
X:Y:Z	X:{Y:Z}

Oh yes now we again left the one language approach :-)