Haxite : writing an entire website using haxe

28.10.2013 31303 4

Here is a screenshot of the draft website, looks like someting I know...

Language management with the haxe.xml.Proxy class :

This is a magic class because some compile-time tricks are done and it's linked to the haxe built-in completion that permits you getting completion for the text identifier stored in an external xml file. This class is well documented here.
In this case I use these texts to display dynamic texts like notifications. The main texts are stord inside the template file (see later)
The Texts class looks like that :

import haxe.ds.StringMap;
import haxe.xml.Parser;
#if sys
import sys.io.File;
#else
import haxe.Http;
#end
class Texts extends haxe.xml.Proxy<"www/assets/texts.xml", String> {
	
	public static var all	= new StringMap<StringMap<String>>();
	public static var lists	= new StringMap<Texts>();
	
	public static function init( path : String ) {
		#if sys
		var content	= File.getContent( path );
		#else
		var content	= Http.requestUrl( path );
		#end
		var xml		= Parser.parse( content );
		parse( xml );
	}
	
	public static function parse( xml : Xml ) {
		for ( tNode in xml.firstElement().elementsNamed( "t" ) ) {
			for ( langNode in tNode.elements() ) {
				var lang	= langNode.nodeName;
				if ( !all.exists( lang ) )		all.set( lang, new StringMap() );
				all.get( lang ).set( tNode.get( "id" ), langNode.firstChild().nodeValue );
				if ( !lists.exists( lang ) )	lists.set( lang, new Texts( all.get( lang ).get ) );
			}
		}
	}
	
	public static function get( lang : String ) {
		return lists.get( lang );
	}
}

This class can be used on server and client side. It checks the identifiers against the path as parameter for the haxe.xml.Proxy class, here "www/assets/texts.xml"
In order to deal with several languages, instead of having a simple "list" variable that is a Text class instance, I use a HashMap first that gives me then a list of xml identifiers and the completion still works fine.
Here is a sample of corresponding xml file :

<?xml version="1.0" encoding="utf-8" ?>
<texts>
	<!-- Errors -->
	<t id="wrongLogin">
		<en><![CDATA[Wrong login or password]]></en>
		<fr><![CDATA[Login ou mot de passe erroné]]></fr>
	</t>
	<t id="invalidData">
		<en><![CDATA[Invalid data]]></en>
		<fr><![CDATA[Les données sont erronées]]></fr>
	</t>
        ...
</texts>

Then I can use it like that :

Lib.println( Texts.get( App.session.lang ).invalidData );

And you get auto completion for "invalidData" ("wrongLogin" and then...) in any IDE that use the haxe built-in completion.

Rounting with the haxe.web.Dispatch class :


Since the haxe.web.Dispatch class is well documented here, I'll not explain all again but just focus on some tricks like meta.
The Dispatch class manages by default for example a missing parameter by throwing an exception "DEMissingParam" or "DEMissing", depending on if it's a form request or not.
Another interesting thing is the meta management.

Meta tags management

When you set a meta tag on a function that is used by the dispatcher's Api, you can then do some kind of generic checks when this function is called.
This is the case of the "adminOnly" meta tag in the documentation example that throws an exception if you'e not a runtime admin (you have to perform the runtime check).
In my case I choose a meta tag named "level" which defines the minimum user's level (a user's paramter) requierd to access the function.
Then I added also a "validPattern" meta tag, that I put on some function that will get arguments, in order to check a valid pattern of the entered text for example.
The meta management for the website looks like that for now in the main class :

enum ApiError {
	AELevel;
	AEValidPattern( args : Array );
}
...
var d    = new Dispatch( uri, Request.getParams() );
try {
	d.onMeta = function( meta : String, mparams : Array ) {
		switch( meta ) {
			case "validPattern"	:
				var errors	= [];
				for ( mparam in mparams ) {
					var r	= new EReg( mparam.pattern, mparam.opt );
					if ( !r.match( params.get( mparam.arg ) ) ) {
						errors.push( mparam.arg );
					}
				}
				if ( errors.length > 0 )	throw( AEValidPattern( errors ) );
				
			case "level"	:
				if ( session.user != null && session.user.level >= Std.parseInt( mparams[ 0 ] ) ) {
				}else {
					throw AELevel;
				}
		}
		
	}
	d.dispatch( new Api() );
}
catch ( e : ApiError ) {
	switch( e ) {
		case AELevel	:
			throw Texts.get( session.lang ).noAuthorization;
			
		case AEValidPattern( args )	: 
			throw Texts.get( session.lang ).invalidData + " : " + args;
	}
}

And here a piece of the Api class :

@level( 10 )
public function doAdmin( d : Dispatch ) {
	d.dispatch( new AdminApi() );
}
...
@validPattern( 
	{ arg : "login", pattern : "[_a-zA-Z0-9]+", opt : "" },
	{ arg : "pass", pattern : ".+", opt : "" }
)
public function doLogin( args : { login : String, pass : String } ) {
	var user	= User.manager.select( $login == $ { args.login } && $pass == $ { haxe.crypto.Md5.encode( args.pass ) } );
	if ( user != null ) {
		App.session.user	= user;
		neko.Web.redirect( "/admin" );
	}else {
		throw Texts.get( session.lang ).wrongLogin;
	}
}

Here we can see that the login and the pass arguments must match a specified pattern defined in the meta tag in order to pass into this function otherwise the "AEValidPattern" exception is thown.

Database ORM using SPOD and DBAdmin :

The SPOD's documentation is also well done, so once again, I'll not detail all but just see some tricks about relations, serlialized objects and the DB Admin project
In order to manage the client session I use the database. The website works also with user accounts. If it's not needed by the final website, it's always useful to have at least an admin user in order to manage all that.
The User class looks like that :

import sys.db.Object;
import sys.db.Types;
class User extends Object{
	
	public var id		: SId;
	public var login	: SString<32>;
	public var pass		: SString<32>;
	public var level	: SInt;
			
	public function new() {
		super();
		level	= 0;
	}
}

As seen previously, working on the web dispatcher and the meta @level( 10 ) : If level >= 10, it's an admin and if user is null or level less than 10, it's a anonymous session for example...
My session contains among others, a "user" field, that is in relation with the User table of course, "ctx" and a "history" fields that are serialized data.
The "ctx" field is used to transmit the template context when a redirection is used, I store there the notifications for example.
The "history" contains a kind of request object with a date, the target uri and the params.
Session class/table :

import sys.db.Object;
import sys.db.Types;
typedef Stamp = {
	public var date		: Float;
	public var uri		: String;
	public var params	: StringMap;
}
class Session extends Object{
	
	public var id		: SId;
	public var sid		: SString<32>;
	public var ip		: SString<15>;
	@:relation( uid )
	public var user		: SNull;
	public var lang		: SString<2>;
	public var ctx		: SNull>;
	public var history	: SData>;
        public function new( sid : String ) {
		super();
		this.sid 	= sid;
		ip		= Web.getClientIP();
		history		= [];
	}
}

All Haxe fields are perfectly mapped to the database structure through to sys.db.Types and thanks to the the meta ":relation( uid )", the database's filed "uid" is linked to the variable "user" that can be directly used. As written here, every changes on user will be applied when session will be commited. Nothing tricky here, but it's really nice to work with database that transparent way .

DBAdmin :

It's a light library used here with SPOD to create the database's tables corresponding to the Haxe classes and then to add easily a light database management application to your website.
I must say that If I like this little lib it is not for its graphical design but more for its lightweight and its easy deployment :
First you install the lib through haxelib, then you add it to the project build.
And here is the code used in the website :

try{
	sessions	= Session.manager.all();
}catch ( e : Dynamic ) {
	sys.db.Admin.initializeDatabase();
	var u		= new User();
		u.login	= "admin";
		u.pass	= haxe.crypto.MD5.encode( "admin" );
		u.level	= 10;
		u.insert();
	sessions	= new StringMap();
	//Web.logMessage( '[haxite] database's tables and default admin account created (pass=admin)' );
}

As you can see, at the begining of the app, I try to get all the sessions in the database. If it fails, I assume that's because the tables are not created so I use sys.db.Admin.initializeDatabase function that will create it for me, with the exact corresponding to the Haxe classes. You just have to create the database by hand. I add also a default admin accout...
Then you just have to add a handler that will "launch" the dbadmin application part.
It's done like that in the web dispatcher Api class :

@level( 10 )
public function doDb( d : Dispatch ) {
	sys.db.Manager.cleanup();
	sys.db.Admin.handler();
}

The constraint to use dbadmin correctly is that it must be accessible from "/db" path. That's why I add it directly in the Api class and not in the sub class AdminApi. But since it has a @level meta, it's secured as the whole AdminApi and only accessible by an admin leveled user
Another thing to note is that since I use dbadmin from my main app (and not independant way by for example building a new index.n in the db directory that uses dbadmin and our 2 db classes...), I have to cleanup the SPOD cache before calling the admin. That's due to some unserialized yet fields as "ctx" and "history" that will be badly understood by dbadmin...

When I wrote this post, dbadmin doesn't compile for the php target, I hope it has been or will be fixed soon.

There are 2 screenshots of dbadmin :

Templating with Templo :

The installation is documented here.
Here is the well done documentation.
So, nothing special here neither except that I've choosen to do as many templates as languages. That's means that the template is special for a language and contains static text inside. Only dynamic texts are variables in the template as seen with haxe.xml.Proxy. You can of course do generic templates and use only templo variables to fill the content.
The template below is the "shell" that will contain sub parts :

<!DOCTYPE html>
<html lang="en">
	<head>
		<meta charset="utf-8"/>
		<title>::TITLE::</title>
		<meta name="description" content="" />
		<link rel="stylesheet" type="text/css" href="/css/default.min.css" />
	</head>
	<body>
		<div id="wrapper">
			<div id="content">
				<div id="info" ::cond INFO != null:: class="::INFO.css::">::INFO.text::</div>
				::raw CONTENT::
			</div>
			<div id="footer">
				Haxite - Powered by
				<ul>
					<li><a href="http://haxe.org"><img src="/gfx/haxe-logo.png"/> Haxe</a>,</li>
					<li><a href="http://nekovm.org">Neko</a>/<a href="http://ncannasse.fr/blog/mod_tora">Tora</a>,</li>
					<li><a href="http://haxe.org/com/libs/templo">Templo</a>,</li>
					<li><a href="http://haxe.org/doc/neko/spod">SPOD</a> &</li>
					<li><a href="http://ncannasse.fr/projects/hss">HSS</a></li>
				</ul>
			</div>
		</div>
		<div id="header">
			<ul class="lang">
				<li><a href="/setLang?lang=fr"><img src="/gfx/flag-fr.gif" alt="French" /></a></li>
				<li><img src="/gfx/flag-en.gif" alt="English" /></li>
			</ul>
			<ul class="menu">
				::if ( SESSION._user != null && SESSION._user.level >= 10 )::
					::set MENU 		= [ "Admin", "Home", "About", "Logout" ]::
					::set TARGETS	= [ "/admin", "/home", "/about", "/logout" ]::
				::else::
					::set MENU 		= [ "Home", "About", "Login" ]::
					::set TARGETS	= [ "/home", "/about", "/register" ]::
				::end::
				::foreach TARGET TARGETS::
					::if ( URI != TARGET )::
						<li><a href="::TARGET::">::MENU[ repeat.TARGET.index ]::</a></li>
					::else::
						<li class="selected">::MENU[ repeat.TARGET.index ]::</li>
					::end::
				::end::
			</ul>
		</div>
		<script src="/js/app.js"></script>
	</body>
</html>

And here comes the context that is transmitted to templo :

ctx = {
	URI		: uri,
	SESSION		: session,
	PARAMS		: params,
	DATE		: {
		fromTime	: Date.fromTime,
	},
	DATE_TOOLS	: {
		format	: DateTools.format,
	}
}
// Get previous context
if ( session.ctx != null ) {
	for ( key in Reflect.fields( session.ctx ) ) {
		Reflect.setField( ctx, key, Reflect.getProperty( session.ctx, key ) );
	}
	session.ctx	= null;
	session.update();
}

As seen before, as I use sessions to pass notifications from page to page. It's a "INFO" variable used in the shell like that :

<div id="info" ::cond INFO != null:: class="::INFO.css::">::INFO.text::</div>

And here comes the admin sub template :

::set TITLE="Haxite - Admin"::
::fill CONTENT::
<div id="admin">
	<h1>Admin</h1>
	<div>
		<a href="/db" class="btn">Database manager</a>
		<a href="/admin/clearSessions" class="btn">Delete sessions</a>
		<a href="/admin/clearCache" class="btn">Clear cache</a>
	</div>
	<h2>Sessions :</h2>
	<ul>
		<li><div style="display:inline-block;width:120px"> </div>
		<div style="display:inline-block;width:300px;text-align:center;font-weight:bold;">SID</div>
		<div style="display:inline-block;width:100px;text-align:center;font-weight:bold;">IP</div>
		<div style = "display:inline-block;width:150px;text-align:center;font-weight:bold;"> <a ::attr href if ( PARAMS.exists( "sort" ) && PARAMS.get( "sort" ) == "user" ) "?sort=-user" else "?sort=user":: > User</a></div>
		<div style="display:inline-block;width:150px;text-align:center;font-weight:bold;"><a ::attr href if ( PARAMS.exists( "sort" ) && PARAMS.get( "sort" ) == "date" ) "?sort=-date" else "?sort=date"::>Date</a></div></li>
		::foreach SESS SESSIONS::
			::if ( SESS._user != null && SESS._user.level >=100 )::
			
			::else::
				<li>
					<div style="display:inline-block;width:120px"><form action="/admin/deleteSession/::SESS.sid::" method="POST"><input type="submit" value="Delete"/></form></div>
					<div style="display:inline-block;width:300px;text-align:center;">::SESS.sid::</div>
					<div style="display:inline-block;width:100px;text-align:center;"><a href="http://www.traceip.net/?query=::SESS.ip::" >::SESS.ip::</a></div>
					<div style="display:inline-block;width:150px;text-align:center;"><span ::cond SESS._user != null::>::SESS._user.login::</span></div>
					<div style="display:inline-block;width:150px;text-align:center;">::DATE_TOOLS.format( DATE.fromTime( SESS._history[ SESS._history.length - 1].date ), "%d/%m/%y %H:%M:%S" )::</div>
					<input type="checkbox" id="cb-::repeat.SESS.index::" /><label for="cb-::repeat.SESS.index::"> </label>
					<ul>
						::foreach STAMP SESS._history::
							<li>
								<div style="display:inline-block;width:120px">::DATE_TOOLS.format( DATE.fromTime( STAMP.date ), "%d/%m/%y %H:%M:%S" )::</div><div style="display:inline-block;width:300px;">::STAMP.uri::</div>
								<ul>
									::foreach KEY STAMP.params.keys()::
										<li><div style="display:inline-block;width:100px;text-align:right;">::KEY::</div> : <div style="display:inline-block;width:100px;text-align:left;" ::cond KEY != "pass" ::>::STAMP.params.get( KEY )::</div></li>
									::end::
								</ul>
							</li>
						::end::
					</ul>
				</li>
			::end::
		::end::
	</ul>
</div>
::end::
::use 'shell.en.mtt'::::end::

Here I use the templates in this way but there are many other ways to play with Templo.
The only things to note here are the way I set "DATE_TOOLS" variable to Templo's context so I can use the "format" function...
Another thing is the use of an underscore to get the "user" from the session or the "history" for example : There isn't any variable named "_user", neither "_history", but this way it means to Templo to use the getter property instead of simply getting the field, so all the stuff with SPOD and then works nice.

CSS design using HSS :

HSS is a CSS pre-processor. See the documentation here
It's a small command line program that eats *.hss like that :

var lightGrey		= #CCCCCC;
var darkGrey		= #666666;
var headerHeight	= 30px;
...
body{
	font-family		: Tahoma;
	font-size		: 12px;
	background-color	: $darkGrey;
	color			: $lightGrey;
}
...
#content{
	padding-top	: $headerHeight + 10px;
	padding-bottom	: $headerHeight + 10px;
}
...
#header{
	position		: absolute;
	left			: 0;
	top			: 0;
	padding-left		: 10px;
	width			: 100%;
	height			: $headerHeight;
	background-color	: black;
	line-height		: 30px;
	
	ul{
		display	: inline;
		margin	: 0;
		padding	: 0;
		li{
			display	: inline-block;
						
			a{
				text-decoration	: none;
			}
                }
         }
}

And you get a CSS file with this kind of output :

body {
	font-family : Tahoma;
	font-size : 12px;
	background-color : #666666;
	color : #CCCCCC;
}
...
#content {
	padding : 10px;
	padding-top : 40px;
	padding-bottom : 40px;
}
...
#header {
	position : absolute;
	left : 0;
	top : 0;
	padding-left : 10px;
	width : 100%;
	height : 30px;
	background-color : black;
	line-height : 30px;
}
#header ul {
	display : inline;
	margin : 0;
	padding : 0;
}
#header ul li {
	display : inline-block;
	zoom : 1;
	*display : inline;
}
#header ul li a {
	text-decoration : none;
}

Tuning with cacheModule & Tora :

This is the original post about cacheModule and mod_tora. I've written a post about that too
Knowing that, and only if we target the nekoVM, we can process some optimizations to speed up the responses of the server.

The neko.Web.cacheModule function :


The first request done by the first user will do our application be cached into memory for next users/requests.
That shows how it works :

static var _tplLoaders		: StringMap;
...
static function main() {
	Web.cacheModule( run );
	//Web.logMessage( '[haxite] module cached' );
	// Lang
	Texts.init( BASE_DIR + "assets/texts.xml" );
	
	// Templo
	Loader.TMP_DIR		= BASE_DIR + "tpl/";
	Loader.OPTIMIZED	= true;
	Loader.MACROS		= null;
	_tplLoaders		= new StringMap();
	run();
}
static function run() {
	Manager.cnx	= Mysql.connect( { host	: "localhost", user : "root", pass : "", database : "haxite"} );
	Manager.initialize();
	Manager.cleanup();
        ...
}
...
public static function getTpl( name : String ) {
	var loader	= null;
	var path	= '$name.${ session.lang }';
	if ( _tplLoaders.exists( path ) ) {
		loader	= _tplLoaders.get( path );
	}else {
		loader	= new Loader( '$path.mtt' );
		_tplLoaders.set( path, loader );
	}
	return loader.execute( ctx );
}

Here as it's written, the static "main" function will be called just once.
The same happen to "_tplLoaders" static variable that is initialized just once and which I use to save the yet loaded templates to do a kind of cache for them.
Templo's loader has got its own cache system, but I haven't found how to "reset" it so I use my own way (In the admin panel I have a "clear cache" button...)
Since when using cacheModule all statics initializations are done just once (including SPOD statics and all...), It's also important to cleanup SPOD at the begining of our application if we don't want to get strange cached values.

Tora Share for the sessions :

This post explain quickly how works the mod_tora : Because the modules are executes from a same multithreaded Tora server, there is a kind of shared objects system. So we can share Haxe objects as Hashes, Arrays or whatever between modules.
Since every request need to manipulate sessions and it's a database call every time, we can use the tora.Share class to replace the SPOD database's session by a Tora shared object.
All you have to do is add the Tora library to the projet and set the mod_tora Apache's module as it would be done for mod_neko. Then just launch the Tora server using this command line :

haxelib run tora 

Then here comes a new Session class version :

class Session {
	
	static var _so	= new tora.Share<StringMap<Session>>( "haxite-sessions", function() {
		return new StringMap();
	});
	
	public var sid					: String;
	public var ip					: String;
	public var user		        	: User;
	public var lang					: String;
	public var ctx					: Dynamic;
	public var history	        	: Array;
	
	public function new( sid : String ) {
		this.sid 	= sid;
		ip			= Web.getClientIP();
		history		= [];
	}
	public static function all( lock = true ) {
		return _so.get( lock );
	}
}

Some changes must be done to the initial code, like the part that get all sessions :

sessions	= Session.all();

And the "session.insert()" and "session.update()" must be removed too since you can directly modify the shared object or its children...
Another thing to say is that if we want to keep the initial template, you remember the trick with the templo "_" to get the getter property, here we have to add a custom getter/setter to the "user" and "history" variables like that :

@:isVar
public var user		(get, set)	: User;
@:isVar
public var history	(get, set)	: Array<Stamp>;
function get_user() {
	return user;
}
function set_user( u : User ) {
	user	= u;
	return u;
}
function get_history() {
	return history;
}
function set_history( history : Array<Stamp> ) {
	this.history	= history;
	return history;
}

Download the sources :

You can download the archive here.
It includes a mixed version thanks to the conditional compilation, that supports both the database session and the Tora session. You can also use the cacheModule or not.
So with this code you can get a php version, a neko one and a special tora one.

Benchmark :

I've done a test using ab from Apache with this command line:

ab -t15 c5 http://haxite/home

And I get these results :

mod_php - no cacheModule - database session
: 38 requests/second


mod_neko - no cacheModule - database session
: 95 requests/second

mod_tora - no cacheModule - database session
: 90 requests/second (1)

mod_neko - cacheModule - database session
: 42 requests/second (2)

mod_tora - cacheModule - database session
: 95 requests/second

mod_tora - cacheModule - share session
: 261 requests/second

(1) : I don't know why the result is less than using mod_neko. Maybe it's due to the module loading ?
(2) : This result could be explained by this post about simultaneous connections and the cost of caching module ?

Commentaires

27.10.2014 à 11:43 relax2code

Hi ,first I want to thank you for making this tutorial and I to ask about the error I got when I want to try this example using nginx and tora fcgi :
Invalid field access : __s
Called from ModNekoApi.hx line 46
Called from Tora.hx line 318
Called from D:\Users\Filt3rek\SDK\haxe/std/neko/Web.hx line 92
Called from D:\Users\Filt3rek\SDK\haxe/std/haxe/web/Request.hx line 65
Called from sys/App.hx line 132
Called from sys/App.hx line 128
Called from ? line 1
Called from C:\HaxeToolkit\haxe\std/neko/vm/Module.hx line 52
Called from Tora.hx line 340
Called from C:\HaxeToolkit\haxe\std/neko/vm/Loader.hx line 154
Called from C:\HaxeToolkit\haxe\std/neko/vm/Loader.hx line 132
Called from Tora.hx line 486

Can you give me suggestion how to run the example ?
Thanks and sorry for my broken english

27.10.2014 à 12:30 Michal

Hej !
It's really a raw example, so It doesn't check anything and the error you get seem to come from here :
var uri = Request.getURI().split( 'index.n' ).join( '' );
Then If you bubble up to where the error occurs, it seems there is a probleme with the uri.
Try to trace( Request.getURI() ); it should be null for you that's why, trying to split a null string into an array it give you that error...
Then try to dig, why it's null.
I'm sorry, I can't tell you more for now because as I wrote you, it's a raw example and I haven't tested it on all environments (not tried on nginx for example...)
Don't hesitate to tell me more directly on my mail (contact@mromecki.fr), I'll be happy to help you

28.10.2014 à 04:25 relax2code

Thank you for the reply , I'll check it again , btw do you know how it can detect D:\Users\Filt3rek\SDK\haxe/std/neko/Web.hx
and D:\Users\Filt3rek\SDK\haxe/std/haxe/web/Request.hx ? , I can't find those classes

28.10.2014 à 06:08 Michal

These are classes compiled into "index.n", and if it tells you my path (compiled on my own computer), so it seems that you haven't compiled the project on your own computer but you use the "index.n" as it.
To begin debugging, first you have to build the project on your own computer (run neko.hxml here if you use mod_neko), it will give you a new "index.n" and then if there is still an error, it will give you your own classpath,

Laisser un commentaire

http://
×