St. on IT

Friday, April 6, 2012

Moving on

Hey folks.

After more than 10 years of being a web developer, I finally decided to make my very own place in the great web

So I'm moving to http://nikolay.theosom.com, all my new posts will be published in there, meanwhile thanks for reading and I'll hope to see you there some day!

Cheers,
Nikolay

Sunday, August 7, 2011

The Lovely Future Of RightJS

Okay, let's make it official, shall we?

Folks, I'm forking RightJS. And before you start screaming like "omg, we all gonna die in here!", calm down, RightJS ain't no going nowhere. The thing I do is future oriented and quite different from RightJS itself. So RightJS will stay where it is, I'm using all the same servers for the fork, so RightJS will be available as long as people use it. And I certainly will apply patches as much as I can.

Just, there will be no RightJS 3.0, RightJS 3.0 will be called lovely.io

So, What's That All Aboot?

Folks, it's not a secret, that monolithic frameworks are dead. Dead as Java. The future is in small, agile and modular libraries and this is where lovely.io starts.

In RightJS we already had kinda modular structure, you could go to the building page and switch off things that you don't need. The problem with this approach is that it is not suitable for the nova-days CDN hosting, all the permutations of a dozen options were blowing the 15k small library into a huge collection of files that weighted several hundreds megabytes and was almost uncacheable. You could host it by uself, but on CDN even if we host it, the chances that another user has the same configurations are slim and it would be pretty much the same as if you host it on your own domain.

Lovely.IO is not really a framework as you might think of a normal javascript framework. It is more like a CDN based package management system. In lovely.io all the RightJS core got split apart on several packages.

* 'core' - basically RigihtJS Class, Events (former Observer), utility functions + some new stuff
* 'dom' - the dom manipulations from RightJS + new jQuery like collections handler
* 'ajax' - new XHR2 based ajax module
* 'cookie' - the cookie module
* 'lang' - JavaScript core extensions
* 'sugar' - syntax sugar for the 'dom' module, things like "something".something() and so on.

Every package is pretty much independent and you can choose what you want. If all you want is 'dom' you just hook it up and it will be just that. If you also like some JavaScript core extensions, then you hook the 'lang' module. Need to do some ajax? Include the ajax module. It's completely up to you what load and what not.

A generic example would look like that

<html>
  <head>
    <script src="http://cdn.lovely.io/core.js"></script>
    <script type="text/javascript">
      Lovely(["dom", "ajax", "fx"], function($, ajax) {
        ajax.load("/some.url", {
          success: function() {
            $('#some-element').html(this.text).highlight();
          }
        });
      });
    </script>
  </head>
</html>

Basically wha happening in there is that you firstly include the lovely.io core package which just 3k of code and provides the modules handling. After that, you can specify the modules you need and once they're loaded (happens asynchronously) your callback function will be called where you can do whatever is it you need to do.

The example above is a bit simplistic. In reality it also automatically handles all internal dependencies, versions and so on. Check out this example, it's a little game, it uses 'dom' and 'cookies', but it also uses another packages like 'timer' and 'hiscore' to function.

This way we are going to have a rubygems like system where anyone can create and share modules, which can relay and reuse each other. All modules including the standard ones will be more or less equal and independent.

Automatic Packages Hosting

Another thing that I'm solving with lovely.io is automatic packages hosting. No more mangling with CDN, demos, documentation and so on. Once you create a package it will be automatically hosted at cdn.lovely.io which sits on Amazon's CloudFront. And it's not just scripts, all your asset images also will be there, automatically and free.

The basic idea is to completely eliminate the process of modules installation. All your package users will have to do is to specify the package name and version in the dependencies, all the reset will be automatically served from the super fast amazon hosting.

Check the mines game I mentioned above. Take a look into the HTML code of the page. It is fairly complex widget, it has some styles and depends on some images, more of that it is a compound widget, the hiscore block is provided by a completely different package. Yet all you have to do to make it working is this

<script type='text/javascript'>
  Lovely(["mines-game-1.0.1"], function(Game) {
    new Game().insertTo('#game-container');
  });
</script>

And it's not all. The micro-modular structure also will provide a big deal of caching optimization. As all the modules are hosted at the same CDN domain, they all will be cached one by one in the user browsers. And because of that the most frequently used packages will be always in cache.

Say a user opens up a page that uses packages A and B, then he goes to a page that uses A, B and C. All he really downloads on the second page is C.

About The Packages

The cool stuff is not over people. Concentrate, because there is more of awesome news!

The other thing is how you actually make the packages. By default there is no JavaScript. Yup, you've heard me. The actual packages code is written in CoffeeScript and SASS (though you can use JavaScript + CSS if you like). Check for example the source code of the mines game from the above. It is all nice and clean now.

How it works? Quite simple. Lovely.IO is not just a hosting. It also has a CLI tool called `lovely` which is much like ruby-on-rails thingy. You can create a project, you can launch a development server and so on.

Then you write your code in whatever you like, and once you're ready do share it with the public, the tool will compile everything in JavaScript, minify using Ugly and push to the lovely.io server. More of that it will automatically use your README and index.html files as landing and demo pages on the server.

You can structure your project as you like, you can have as many files as you like. You can use CoffeeScript or JavaScript, SASS or CSS, or even Stylus. It all doesn't matter for the end result.

Finally you can do the front-side development in a clean and civilized way.

Other Stuff

There is one thing among this awesomeness though. Lovely.IO is an HTML5 oriented thingy, there is no IE < 9 support, not like I couldn't, just it's thankless job to support those old browsers. I don't wanna do that, you don't wanna do that, and the guy next to you don't wanna do that either.

So, in lovely.io there will be no old IE browsers support in the standard packages, plus there is a good chance that I'll be nuking any other packages that will try to provide the old browsers support explicitly.

Basically, this is where I'd like to draw the line between RightJS and LovelyIO. If you need something well tested with old browsers support, use RightJS, it will be around as long as you want it. But, if you're building a modern HTML5/CSS3 application then you might want to consider LovelyIO, this is where all new stuff will go.

Current Status and Future Plans

At the moment Lovely.IO has all the modules from RightJS core + RubyOnRails support module + I've made a bunch of demo projects over here. It appears to be more or less stable (it's based on RightJS after all), but there is no UI yet.

There is also not much of documentation either. I've just opened the registration for early adopters. So if you wanna play with it, please check the main repository, this is where the STL packages live. It's mostly the good old RightJS with a bit of lean towards more standard jQuery behavior for the $ function.

The next step in the project will be automatic documentation hosting for the packages and then I'll start to port UI modules from RightJS, so if you want me to port something specific from RightJS UI, let me know I'll try to make it a priority.

--
Okay, I think that's the whole introduction for now. I'll give you more when I get to the documentation.

Saturday, May 28, 2011

Calculating Cubic Bezier Function

If you ever played with the CSS3 native css-transitions, you know that they use cubic-bezier notation to define the transition formula. Cubic Bezier is a parametric function that allows you to define all sorts of smooth curvatures that looks kinda like that

The reason why it's used instead of sinuses and exponents, is that it allows you to define all sorts of shapes with just a few more or less intuitively understandable parameters. Plus, unlike other mathematical functions it can be computed very fast, because it's a simple parametric function, no sequences no nothing.

In RightJS we support native css3 transitions, but it also has a pure JavaScript fx-engine to handle old browsers. And the thing is that native CSS3 transitions use the cubic-bezier functions, while the pure JavaScript engine uses a bit of mumbo-jumbo with Math.sin() and Math.log(), and what I wanted is to allow the developers to use cubic-bezier notation with the old engine.

The problem is that it's not that simple. The cubic-bezier function itself is pretty much straightforward and you can calculate a set of X and Y coordinates in no time, but it won't make you any good because both x-s and y-s will be related to the third t parameter and therefore unevenly distributed agains the t parameter. What you really need is a straight Y = B(x) function.

And here it gets tricky. That much tricky. But, luckily for us, there is always a better way to do it. All you need is to recall a bit of kindergarden math.

Originally, the cubic bezier function is used in a parametric form, but it also can be converted into a polynomial form, in which case it will look like that (you can google it and find say this document)

a = 3 P0
b = 3 P1
c = 3 P2
c0 = P0
c1 = b - a
c2 = a - 2b + c
c3 = P3 - P0 + b - c

B(t) = c0 + t(c1 + t(c2 + tc3))

In case of CSS3 cubic-bezier definitions, P0 is always 0 and P3 is always 1, in which case the formula gets even simpler

c3 = 3 * P1
c2 = 3 * (P2 - P1) - c3
c1 = 1 - c3 - c2

B(t) = t * (c1 + t * (c2 + t * c3))

Which, in case of of CSS3 x1,y1,x2,y2 parameters, can be written in javascript with two functions, for example this way

var Cx = 3 * x1;
var Bx = 3 * (x2 - x1) - Cx;
var Ax = 1 - Cx - Bx;

var Cy = 3 * y1;
var By = 3 * (y2 - y1) - Cy;
var Ay = 1 - Cy - By;

function bezier_x(t) {
  return t * (Cx + t * (Bx + t * Ax));
}

function bezier_y(t) {
  return t * (Cy + t * (By + t * Ay));
}

There are two reasons why we use the polynomial form of the equation. Firstly, it has less operations and therefore will work faster. Secondly, we are going to use Newton's method to approximate the values and will need a first derivative of one of the functions, which in case of polynomial functions is pretty simple.

Now, to the fun part. We converted the function from one form to another, but the question remains the same, how are we going to get the y = B(x) from x = B(t) and y = B(t)? Quite simple my friends, we will use the kindergarden math, or more exactly the Newton's method to approximate a parametric x value for every t value so that we could get an evenly distributed set of x values and then get a set of y for them using the same bezier_y(t) function.

Newton's method might look a bit scary for those who are not familiar with it, but it is actually very simple. Check this video the guy explains it nicely.

All you need to make it work is to have a derivative to your function, which in our case will look like that

function bezier_x_der(x) {
  return Cx + t * (2*Bx + t * 3*Ax);
}

And after that, we just define a little function that will make several iterations trying to find an x value which came from the current t value. We don't have to be very precise in our case, we just make 5 iterations tops and limit the precision by the 3 digits after the pointer.

function find_x_for(t) {
  var x = t, i = 0, z;

  while (i < 5) {
    z = bezier_x(x) - t;
    if (Math.abs(z) < 1e-3) break;

    x = x - z / bezier_x_der(x);
    i++;
  }
  
  return x;
}

Now you can create a loop and calculate a set of evenly distributed values, just like that:

var result = [];
for (var i=0; i < 1.001; i+=0.1) {
  result.push(bezier_y(find_x_for(i)));
}
console.log(result);

That's basically all of it. You can find the full version of this script in this gist.

Enjoy!

Sunday, May 8, 2011

How To Read User Input With NodeJS

I think it's time for me to join the NodeJS hysteria. So, I'd like to share a bit of things I was tackling today because there is not really much of docs on that matter right now.

Short things short. If you're writing a little console script in NodeJS that supposed to ask the user this and that, here's how you can get it done.

Node's global object process has two properties called .stdin and .stdout, which are essentially streams. You can write things into the stdout and listen to the 'data' event in the stdin stream. A simple example from the api-docs looks like that

process.stdin.resume();
process.stdin.setEncoding('utf8');

process.stdin.on('data', function (chunk) {
 process.stdout.write('data: ' + chunk);
});

You need to call that .resume() method first, it initializes the STDIN reading process.

It is pretty straightforward, but it's almost useless when you need to ask the user several questions, more of that validate the input data and re-ask questions when something was entered wrong.

The basic trouble here is that .on('data' handler that will fire every time the user hits the Enter, so you'll have to figure out which question is currently asked an where to put the data. Given the fact that everything is happening asynchronously, it pretty quickly gets really messy.

Fortunately for us, there is another events handler in NodeJS called .once(.. it's basically same as .on( but it detaches the listener after the first event received. Knowing that, we can write a little helper function like that

function ask(question, format, callback) {
 var stdin = process.stdin, stdout = process.stdout;

 stdin.resume();
 stdout.write(question + ": ");

 stdin.once('data', function(data) {
   data = data.toString().trim();

   if (format.test(data)) {
     callback(data);
   } else {
     stdout.write("It should match: "+ format +"\n");
     ask(question, format, callback);
   }
 });
}

There are two important moments. First of all, don't use console.log to ask the user the question, otherwise it will print a new line at the end, which is kinda sucks. Secondly note that you should call .toString() and .trim() on your data, the first one to convert the data from a stream into an actual string, the second one is needed because the string will have a trailing new line symbol at the end after the user hits the Enter key.

After that, you can ask the user all sorts of questions. For example like that

ask("Name", /.+/, function(name) {
  ask("Email", /^.+@.+$/, function(email) {
    console.log("Your name is: ", name);
    console.log("Your email is:", email);

    process.exit();
  });
});

And don't forget to call the process.exit() when you done, otherwise the script will just hung in there.

That's basically all of it. Enjoy!

Thursday, April 28, 2011

Why Make A Run For It

It has been almost two years since I decided to go public with RightJS and on this occasion I'd like to post some reflections on it.

Those days, thanks to github, it's pretty simple to start an open-source project. You just make a repo, write some awesome stuff and push it under the "I don't owe you shit" license. And I like it, in most cases that's all you really need. But, as fun as it is, there is actually another level to open-source projects. The level, where you decide to get serious about it and go public with your project. With promotions, users, official site, docs, tutorials and so on.

It is not completely necessary to make your project kinda "officially public", in many cases it might be even a bad idea, because that will distract you from the actual programming, but there are several important things in this process, which can be pretty valuable as well. And that's what I'd like to write about today.

It Will Make You A Better Programmer

The trouble with programmers is that we work in this area where most people don't really understand anything. And because of that, at some level, programmers tend to start thinking that they are pretty darn smart and kick ass.

A healthy portion of self-esteem is not a bad thing, it might help you to overcome some small obstacles on your way, but for a programmer as a knowledge worker it is crucial to stay hungry for new information.

And running your project for real, will give you exactly that. If you do something interesting enough, people will respond, they will throw all sorts of ideas at you, they will criticize and they will like (hopefully :)) some parts of your work, which will give you all sorts of new information.

With new information you'll face new problems and discover new ways to solve them, and that will make you better at what you do.

It Will Make You More Organized

It is not a secret that many developers don't like TDD and Tickets. Even the ones that do, often cheat and don't cover everything, don't document bugs and so on.

But, when you start an open project for real, you will have to deal with those things. You will have to handle tickets, document bugs and cover everything you can with tests so that folks who use your code had a hight quality experience.

More of that, most open-source projects, especially individual ones, are just about "scratching your own itch". It is highly doubtful that you will make any real money directly from your project, and as much as you might enjoy the creative process of writing code, at some point, you will have to get pragmatic about what you do.

You will need to start thinking strategically, where is it worth to invest your time and where it's not. And that is a highly valuable skill, not just for developing a software project, but genuinely in life.

It Will Make You Better With People

Well, you know what they say "all programmers are bad with people" :). The thing is, that people are irrational, even the smart ones. And there is only one way to learn how to deal with irrational - by accumulating experience.

One of the hazards of the trade is that programmers spend most of their time with computers, and that's certainly not a good thing for your social skills.

When you run an open-source project you will have to handle contributors and do that nicely. Because contributors are privilege. But as all people have different coding styles, different ways to approach issues and just different personalities, you will learn how to find common grounds with different folks, how to work together and how to coordinate the changes.

Conclusion

I don't want to sound like an open-source project is the most awesome thing that might happen to you. By the end of the day, if you're vicious bastard then all that will help you is an old-fashioned kick in the ass.

But, if you're interested in software-development, if you want to learn why and how the things done, running your own project for real might be very insightful and teach you quite a few things.

Monday, April 4, 2011

Making Tags With Rails And RightJS

As the tags widget appeared recently on the RightJS UI list, I'd like to write a simple how-to about tags in case of using RubyOnRails.

You can see the tags widget in action on this demo page, and over here you can find a demo rails application that has the tags feature implemented, along with some other right-rails features.

Let's start

The Initial Model

When you need to add a tags feature in a ruby-on-rails project, a standard model would look kinda like that

class Article < ActiveRecord::Base
  has_and_belongs_to_many :tags, :uniq => true
end

class Tag < ActiveRecord::Base
  has_and_belongs_to_many :articles, :uniq => true

  validates_presence_of :name
  validates_uniqueness_of :name
end

As you can see, it's a pretty much straightforward m2m association, articles have many tags and tags have many articles.

Tags To Strings Conversion

The next standard step is usually to add a couple of methods to convert the list of tags into a coma separated string back and forth. The basic reason is that tags are data-base records and it's not much of user-friendly to use integer ids

class Article < ActiveRecord::Base
  has_and_belong_to_many :tags, :uniq => true

  def tags_string
    tags.map(&:name).join(', ')
  end

  def tags_string=(string)
    self.tags = string.split(',').map do |tag|
      unless tag.blank?
        Tag.find_or_create_by_name(tag.strip)
      end
    end.compact
  end
end

After that you can use this proxy property in forms directly, like that

<%= form_for @article do |f| %>
  <p>
    <%= f.label :tags_string %>
    <%= f.text_field :tags_string %>
  </p>
<% end %>

This way the user will always manipulate the tags list as a simple string and your app will automatically create the tag records and associations on fly

Hooking Up RightJS

At this point your rails app will present a kind of a standard and simple case of tags handling. Time to make it look nice and friendly with a bit of javascript.

To hook up RightJS into your Rails application all you need is just add the right-rails gem into your Gemfile

gem 'right-rails'

And run the following code generator

rails g right_rails

Don't worry when you see a tall list of scripts it copies into your public/javascripts directory, right-rails handles most of them automatically, so you won't really need to look inside of those.

Adding The Tags Widget

Ironically, the actual RightJS part of this enterprise is the shortest one. Once you added right-rails into your application, all you need to do is replace 'f.text_field' with 'f.tags_field' in your form

<%= form_for @article do |f| %>
  <p>
    <%= f.label :tags_string %>
    <%= f.tags_field :tags_string %>
  </p>
<% end %>

RightRails handles everything else automatically, it generates all necessary html, includes javascript/css modules, it even switches between minified and source builds of javascript libraries depending on the working environment.

More of that, it can automatically use the google hosted rightjs cdn-server for a superfast, shared scripts delivery.

Adding Autocompletion

By default the f.tags_field will make you a pretty looking tags-field, if you want to add the tags autocompletion feature to it, all you need is to specify a list of known tags in it

....
  <%= f.tags_field :tags_string, :tags => Tag.all.map(&:name) %>
....

There is also a few additional options you can use with tags widgets, you can find them at the rightjs.org documentation

Conclusion

This is a good example to show how RightJS is build with server-side developers in mind. As you have noticed we didn't write a single line of javascript in our little exercise, not like we couldn't, just we don't have to bother with those routine things every time we need a simple form with a bunch of standard widgets.

More of that, if you take a look into your HTML code, you'll see that RightRails didn't write a single line of script either, all it did is added the data-tags attribute to your input field.

<input data-tags="{tags: ['one','two','three'}" value="one, two"
  id="article_tags_string" name="article[tags_string]" />

This way, even if something will go wrong, the user will still see the standard input field with coma separated tags. All your styles and content will remain were it was.

--
Well, this is pretty much the whole story. If you're interested, go check the RightJS UI collection, it has many more useful widgets that you can use in your applications.

Thursday, March 24, 2011

Why ==null, Not ===null

Sometimes in RightJS source code you might see a thing like this

if (value == null) {
  // ...
}

And as every JavaScript optimization tutorial tells you to use '===null' instead of '==null' and JSLint yells at you when do otherwise, people immediately try to change it to the "right" way.

Meanwhile there is a pretty good case when it is much better to use '==null' than '===null'.

The thing is, that in JavaScript we have this ugly thing called 'undefined', which is quite clunky and often weird in use. Like say you might have a piece of code like that

var hash = {a: undefined};
console.log(hash.a); // -> undefined
console.log(hash.b); // -> undefined

console.log('a' in hash); // -> true
console.log('b' in hash); // -> false

Which is a bit strange. So, it is quite often, people use 'null' to predefine keys in hashes and then need to check whether a certain value is not 'null' or 'undefined', kinda like this

if (value !== null && value !== undefined) {
  // ...
}

Another case is that the 'null' thing is an object in JavaScript and sometimes you might need to check if a variable is a plain type value, you go with things like that

if (
  typeof(value) !== 'object' &&
  value !== null && value !== undefined
) {
  // ..
}

There are lots of cases where you need this double check. You also should keep in mind the fact that 'undefined' is a variable in JavaScript and someone actually can define a variable named 'undefined', and in general case you'll need to handle that too. Well, you get the ugly picture.

So, what about that '==null' business anyways?

The trick is that '==null', unlike other boolean comparisons (for example '==true'), returns 'true' only for 'null' and 'undefined' values.

null == null;      // -> true
null == undefined; // -> true

null == false;     // -> false
null == '';        // -> false
null == 0;         // -> false

Which basically means that you can use value == null instead of

value === null || value === unefined

More of that, except being more compact, 'value == null' also will be protected from 'undefined' replacements, and the browser also won't perform a variable name lookup, because null is a keyword and there is nothing besides it. Which will make the things faster.

How much faster? Let's see. I cranked this little script and run it in FF4

function test1() {
  var i=0, t = new Date(), v1 = null, v2 = undefined, r;
  for (; i < 100000; i++) {
    r = v1 === null && v1 === undefined;
    r = v2 === null && v2 === undefined;
  }
  console.log(new Date() - t);
}

function test2() {
  var i=0, t = new Date(), v1 = null, v2 = undefined, r;
  for (; i < 100000; i++) {
    r = v1 == null;
    r = v2 == null;
  }
  console.log(new Date() - t);
}

console.log('--');
10..times(test1);
console.log('--');
10..times(test2);

And here is the result

Well, that's the whole story. Think you can put it in a good use :)