Building a 680 Megapixel image with Processing


For the last 4 months I have been working on a Game project with Processing and Processing.js at the Centre for Development of open Technology here at Seneca College.

This game is designed to help teach the basics of Mendelian Genetics to high school aged students and above.  The concept is to collect and breed different dragons and learn how the genetics of it all works because the offspring are created based on the genetics of the parents.

One of the challenges I ran into when building this game was figuring out a way to handle all the different combinations of dragons.  The dragons have 9 genetic traits that differentiate them from one another and each trait has two or three different phenotypes associated with it.  The end result is there are 2592 different dragons that can be created and collected in the game.

These dragons were being dynamically built from a combination of 27 smaller images and then blended together, colorized, and then had patterns applied to them.  In the native Processing JAVA environment this process took only about 100 milliseconds to build each dragon.  But in the Processing.js environment this process took 700ms on Chrome, 1000ms on Firefox, 2500ms on IE9, and a whopping 4500ms on Safari.

This created a lot of problems, such as slow script warnings, when there were more than 4-5 dragons to build on game load.

To solve this problem I created a sketch in Processing to dynamically build every combination of dragon and call save() on it from a PGraphic to save out an image for every dragon in the game.  Now with an automated build process I can quickly (roughly 5 minutes) generate all 2600 dragons into images and load them up with loadImage() or requestImage() in Processing.js significantly improving the load times when starting the game.

As a side challenge to this sketch to generate all the images I added some code to build a massive collage to stick in a copy of every dragon in the game in a giant PGraphics image.

PGraphics massiveImage = createGraphics(27648,24576,JAVA2D);

Immediately I had problems with this with not enough memory being able to be allocated to the JRE. Because java runs in a 32 bit environment for the PDE I could only allocate about 1600mb of ram to the environment. This meant I would have to create eight separate images and then composite them later in Photoshop unfortunately.  I was still able to do it though and through some crafty code I was able to make eight images that were 6144×13824 in size and then patch them together into a giant 680 megapixel composite image showing all 2592 dragons at 512×512 resolution.

reduced size collage of all 2592 dragons Generated with Processing

Compositing the image in Photoshop took 11 GB of ram, 679477248 pixels, and created a 60MB PNG image file in the end.

If I had been able to allocate 4-5GB of RAM to the PDE I think I could have generated the entire image in one shot and I would have liked to have been able to add a sitting and flying image of each dragon bumping up the resolution to 1.35Gigapixels in total.



Using IIS7.5 with Dropbox to host websites


Up until recently I have been developing a web project using Processing.js and hosting it locally using IIS for development and testing.

For those who aren’t familiar with Dropbox it is a service that allows you to store 2GB of files in the cloud on Amazon’s S3 service for free. It also has Windows and Mac clients that sync files from a special folder on your hard drive to the cloud. The service is super simple to use and the integration with a local folder on your PC makes it extremely easy to work from multiple locations or computers with the same set of files because you can sync your files to multiple computers or with multiple Dropbox users by sharing folders.

When you combine this with the power of GitHub and a web server you can create a system that allows you to develop a web project, have it hosted from that folder on the web, and sync all the files between multiple machines as well as check in files to a managed repository cloud for others to work as well.

I can work from home and all my files are automatically synced from work, when I change a file at home it automatically syncs the file to the web server for me and because IIS points to the synced folder to serve to the web it means as I develop remotely my web server is always up to date within a few seconds of saving a file and the website is updated as well.

Recently though we had a power failure at Seneca and something caused IIS to throw permission errors when linking a virtual directory to the Dropbox folder containing my website.

It turned out I had to add the ‘Authenticated Users’ to the list of authorized accounts that can access the folder. I am also using Basic Authentication for password protection on the website for a quick and easy way to block anonymous access to the website.

The basic steps for creating a password protected development website that you can sync all your files between machines goes like this:

  1. Install Dropbox on all computers you will be working from including your host web server.
  2. Setup your website folder but do not share this folder with other people to modify. This is bad practice you don’t want others changing files on your website from a folder on their computer. Use a source control repository like Github to manage code.
  3. Install IIS on your host machine.
  4. Verify the port your using to host websites is open (usually port 80) on your network.
  5. Create a user for people who will be accessing the website remotely on the host web server.
  6. Create a virtual directory in the default website and set the alias to whatever name you want your users to access like mydevsite which will be accessed by myserveripordomainurl/mydevsite.
  7. Set the physical path to point to your Dropbox folder.
  8. Click on ‘Connect As…’ and check ‘specific User’, then enter a user that has permissions to access this folder and their password.
  9. Click ‘Test Settings…’ and verify both checks pass ok.
  10. If you have a domain name point it to your server ip or create a CNAME record that points to your IIS server ip to make a friendly url for those accessing your website.
  11. In IIS manager, select your virtual directory and pick ‘Authentication’. Disable Anonymous Authentication and enable Basic Authentication.
  12. Ideally you should Enable SSL and use HTTPS and also use a better authentication method than Basic Authentication but that is beyond the scope of this checklist.

Continuation with Seneca


It’s been a busy first four months of 2011 for me finishing up my last semester for my Bachelors degree in Software Development at Seneca but it appears to be worth it.

Now that I’ve graduated I’ve taken a position with the Office of Research and Innovation at Seneca in the Center for Development of Open Source Technology for the next year working on some projects with industry partners.

Some of them will use Processing.js in part and others will be other technologies. At this point that’s about all I can say though.

One of the things thats been holding me back from blogging besides limited time availability has been the fact that I’m waiting on one last grade to officially _officially_ be able to state that I am graduating.

Part of the reason for this was a final project I have been working on for a professor that includes two processing sketches for use with his course website to help teach game programming students the basics of vectors and matricies. Specifically how vector notation works and what the notation means visually.  The sketch shows how a vectors coordinates change. As you move the head of the vector around it updates the vector notation in realtime and shows the angle of theta. The second sketch is a matrix multiplication calculator that lets the user type in values for two 4×4 matricies and then press the calculate button to see the result of multiplying them together.

I used standard HTML input boxes and then with JavaScript and Processing.js I take the numbers and multiply them out and show the result in a matrix on a canvas.  Sure I could have done it in pure JS and made it simple but I wanted to make use of Processing.js and the canvas element so that I can add more functionality in the future that shows an animation of how the math works to get the result for each piece of the matrix.

This week I continue work on a few projects here at CDOT. One being a game using Processing.js and another being a dashboard app that will show status of the projects going on at CDOT.

Firefly + NAS + iPhone + iTunes = awesome


I recently purchased a network attached storage server for home for a bunch of different reasons. Mainly so I would be able to backup my important files redundantly, store all my media for the entire network to have access to 24/7 without leaving my PC on, and to eventually allow me to switch my PC over to a SSD and just have applications on my pc.

So I pickup a Netgear ReadyNAS NV+ from Canada Computers on boxing day and it was the best purchase I’ve made in a long time.

It is so much more than a file server with RAID.

  • iTunes streaming media server
  • DLNA streaming media server
  • Squeezebox streaming service
  • Bit Torrent web client
  • FTP server
  • CIFS/SMB/NFS/AFP file sharing (WIN/MAC/XNIX)
  • HTTPS web admin like a router
  • Remote access from the internet
  • SSH remote access capabilities

And most of this can also be port forwarded from your router so you can access it anywhere on the internet. Add a subdomain DNS record and you have a friendly URL to connect to it all too!

So tonight I spent a few hours setting up the Firefly open source media server that it comes with which allows you to stream music from the NAS to iTunes as a share and selectively play any songs you want as if the music was stored locally.

Sure you can say oh I could just connect iTunes to a shared folder on another computer and do the same and yes you could but can you build a computer with RAID5 and 4TB of space for under $500? You’ll spend $300 on the HDD’s alone.

So after I got the iTunes server up and running I started thinking how can I stream to my iPhone now and also I wonder if the apple remote app will work to control my iTunes server now.

The answer was yes to both!

You can grab the Apple Remote app from the app store and connect to your wifi network and authorize your iPhone/iPad/iPod to control iTunes on your PC.  You can then access the iTunes streaming server and have full control over iTunes. You could build a HTPC and connect it up to a home theater and then have wireless control of your music from iPhone with all your media redundantly stored or just store it on the HTPC as well.


iTunes Server

iTunes Server Playlist

Now if you want to actually stream your media to your iPhone instead of just control your PC with the iPhone you can do that too.  A couple apps I found that can connect to DLNA streaming devices are MLPLayer and 8Player.  Both have lite versions for free to test out.  8Player works over 3G as well so you could port forward your DLNA service out on the web through your router and have streaming music access anywhere in the world with a 3G connection on your iPhone. Not just your music but also movies and photos can be played with these apps.  I streamed a 2.3GB movie over my wifi to my iPhone seamlessly and it worked perfectly.

8Player browser

8Player movie selection

MLPlayer lite browser

District9 in HD streaming (MLPlayer)

Subtitles supported while streaming from DLNA

Update Jan 7 2012: I’m now using 8Player on iPhone and iPad. Best app purchase I’ve ever made on the app store. In 8Player the server URL you will want to specify is http://youripordnsname:8200/rootDesc.xml. Default port for the service is 8200, but you could map it with your router to any external port to 8200 internal if your router is more sophisticated.


Updated Processing.js reference


It took me a few days to do it but I managed to give the Processingjs reference a much needed update to fix a plethora of broken links we had in the documentation. All links should point to the right location now and links between types and parsing functions shouldn’t be mixed up anymore.

I’m also working in a new piece for the reference to document the different Processingjs directives that can be used. There are things such as anti aliasing, transparent backgrounds, and image preloading that we need to document on the processingjs site. Currently this documentation exists out on the web but its scattered.

My job for the next few weeks is to begin moving it all into a central place either on the reference pages or in a wiki which we are in the process of setting up as well.

Stay tuned.

Processing.js 1.0 released!


It’s been a long time coming but the Processing.js team has finished up the last of the tickets for 1.0 and we are proud to release Processing.js 1.0.

You can download it here.

This release includes a ton of features which have spanned through the internal 0.9.8 release and the 1.0 release combined.  You can read the changelog here.  We closed over 100 tickets for this release and spent a considerable amount of time on documentation, tutorials, website updates, and other user and community support infrastructure to make it easier for the end user to experience Processing.js.

If you haven’t had a look at the Processing.js website lately you will see that the exhibition page has been updated with some new examples of Processing.js in use around the web.  The learning tab has been updated to show some new tutorials on how to use Processing.js and some new IDE’s that have popped up on the web over the last while.

Check out the new exhibition entries and the new interactive development environments.

The reference pages have also been updated by Anna over the past week or so for new users and old who want to look up information on the language.

Another major milestone for this release is to provide some initial support for the IE9 beta.  It’s got 1 or 2 issues which we have bugs filed on but for the most part it works very well.

In terms of speed this release has also improved significantly in some areas.

Try it out and show us your work around the web!

Preparing for the release of Processing.js 1.0


I haven’t written about Processing.js in a long time so I though it might be time to post something new with the release of 1.0 tentatively coming up in a few weeks.

We are hoping to finish up the remaining code related bugs and additions within a few days and get the 1.0 release up onto Corban’s test suite to begin testing before finally releasing 1.0.

If you are interested in helping out you can signup on the test suite site and help us test out the release of 1.0 on multiple browsers and platforms. Earn points on the leader-board and work your way up to the top by helping us test the latest release of Processing.js!

If your new to Processing.js and haven’t heard about it…

Processing.js is an open programming language for people who want to program images, animation, and interactions for the web without using Flash or Java applets.

Here are a few projects and tools around the web that use Processing.js that show what it can do.

We are also planning a game development in Processing.js workshop at FSOSS being delivered by Andor Salga on Oct 28 at 9AM in room T2109 at the Seneca @ York campus, TEL building

If your interested in the status of the upcoming release or want more info on Processing.js you can stop by the active IRC channel on channel #processing.js

How to build a class diagram for your projects in Visual Studio 2008/2010 automatically


One of the great features of Visual Studio is it’s ability to easily create diagrams showing the relationships and inner workings of your classes for applications you are building.  Visual Studio allows you to create class diagram files (.cd) in all kinds of different projects such as visual c++ applications.  This diagram file has a design view interface that allows you to drag and drop classes from your solution explorer or class view.  The display will show the class with all its properties and methods and which access level they have.  It will also show relationships between classes and inheritance relationships.

Soundcard Class

Here is an example of a class showing in the design view

From this you can see a number of useful things.  You can see that the class name is SoundCard, it inherits from the iSoundCard interface, and it has a number of fields (properties) and methods (functions).

The permissions of the fields and methods are represented through the images to the left of each field or method name.  Here is a summary of the main ones

  • Private const field
  • Private field
  • Public method
  • Private method
  • Protected method

There are a number of other features the class diagram has such as showing inheritance, macros, structs, and allowing you to save the diagram as an image.

To create a new class diagram just choose Add > New Item on a source folder you want to place it in.

Then select Class Diagram

Then you will be able to create a full class diagram from your classes in your project.

Now I’m sure the next question your asking is what if I update my class or remove it or add a new class ?

Well when you make changes the diagram automatically updates itself for you.  All you have to do is print it again or re-save the image if you are using it outside Visual Studio.

Here is an example of a fully formed class diagram based on the framework we work with in our Game programming course at Seneca.  This framework was written by Chris Szalwinski and is used by GAM666/DPS901 students.

New wordpress host


I decided to finally do something with my domain that I purchased a while back and after thinking about it it seemed like a WordPress install with its built in admin, page, link, blog, theme, and plugin support would allow me to create a rich personal website to eventually host my own online portfolio with.

I wasn’t wrong so far.  In less than an hour I had configured my host, setup my domain, installed WordPress, created and configured the database, configured my theme, installed a few plugins, and imported my blog posts from my old WordPress hosted blog site.

Over the next few weeks I’ll continue to tweak it and play around with themes and CSS and explore the thousands of available widgets and plugins.