Website Still Down

The Render Engine website it still down.  I've spoken with my hosting company and they tell me that the machine died but that the site data is safe.  I'm not sure when the site will be live again.  Progress on v1.5.4 has been hindered by this, and other factors (namely the holidays), but I intend to get it finished as soon as I possibly can.

Dirty Rectangles III

The website is still down - Not sure when it will be up again...


More time has been spent on dirty rectangles to try and eek out some performance during rendering to canvas.  There are basically two methods which I am currently fighting with:

1. Brute force method
2. Joined rectangles method

The first method would go through the collection of objects, and for each dirty object it would grab an image of the space behind it and store that.  For the next frame, it will restore those stored rectangles and then the process starts again.  Objects which move, but then stop, would need to update the layer to which they draw to keep this process somewhat automatic.  Otherwise, objects that stop being dirty would be lost during the next frame redraw.  The second method would use something similar, but instead of capturing an image for every dirty object, it would combine rectangles which overlap.  The difference between the two is timing.

With the first method, we just go ahead and grab dirty rectangles for each object regardless of whether another object is grabbing that rectangle (or part of it) as well.  Thus, if there are 30 dirty objects, we grab 30 rectangles each frame, draw 30 objects, then for the next frame we restore 30 rectangles and do it again.

If we combine rectangles, it might add up to a savings if the objects overlap.  However, time will be spent determining the overlap and joining rectangles before actually grabbing the images to later restore.  I'm not sure if the trade-off is worth it without actually testing it.  The actual method is pretty intensive and requires visiting objects, which have already been visited, repeatedly.  This might end up causing more overhead than just a simple brute force method.

The other issue, with dirty rectangles, is the particle engine.  Capturing hundreds of tiny little rectangles for the particle engine is messy, and time consuming.  The actual time saved is lost because we're doing twice the work for all of those particles.  For larger objects it makes sense.  If we can eliminate the need to redraw 30 objects, if only 3 are moving, then we save time.  There are some tricks for particles, such as using bitmaps for the explosions which simulate particles but that might look bad.

Maybe using the Asteroids Evolution demo to test all of this isn't the best idea, after all.

Dirty Rectangles II

The site is currently down.  I've sent an email to my provider and hope to know something shortly.

Yesterday I was working on a dirty rectangles method.  What it has boiled down to is something similar to a SpatialGrid, whereby I subdivide the context into smaller blocks and determine which ones contain dirty objects.  A dirty object would be anything that would change how an object is rendered: line style, points, position, rotation, etc.  Additionally, I'm using the z-index to group objects together into "layers" which are rendered together.

Object2D has the idea of a z-index which allows us to position objects in front of, or behind, other objects.  Using this z-index would allow me to organize objects into rendering "bins" and perform updates for each bin as a whole.  So each frame that is generated would:

1. Go through the bin and see which of the rectangles contain dirty objects
2. Capture the contents of those rectangles (the dirty rectangles), and 
3. Draw the objects for the bin.  

Each bin would process separately, so it would be possible to have some background layers (objects that don't change very often), and active layers where things do change often.  Then, on the next frame we would restore those dirty rectangles, and start the process again.  This should reduce the number of times that we need to reset the entire context before redrawing, which in turn reduces the amount of drawing that must occur each frame.

Additionally, each object that isn't dirty doesn't need to be redrawn because it hasn't changed in any way, further reducing the number of drawing calls that must occur.  Objects still need to be processed, but most of the time is spent rendering the objects anyway.

Tonight I plan on implementing this method.  If anyone has any input for this method, let me know.  I've already considered sparse rectangles, but the simplest (and quickest) way I figure it could be done is via subdivision of the viewport.

v1.5.4 Coming

There are some things that have been going into v2.0 that would just make sense to get into a new point-release. So, I'm going to get a v1.5.4 together and tested.  The new stuff is mainly some new functions in Math2D and some updates to the math primitives, all regarding transformations and stuff along those lines.  No new features, per se, but something that would be useful to people currently using the engine.

Asteroids and SAT

Well, it's been about a week since I last posted something so I wanted to get a new entry out to let people know I'm still actively working on the engine.  As those who follow my posts may know, I've been working on getting a real narrow-phase collision system into The Render Engine for a while now.  Tutorial 11, in v2.0, is a simple example of using SAT.  However, I wanted to really push the extreme and get a good test for the new SAT convex collider, collision hulls, and fix up a lot of part of the engine.  This stuff is all new, but it's finally to a point where I can show it off:

Asteroids with SAT

The full game isn't working, there are bugs, but at least the attract screen is showing promise.  All of the asteroids are using the convex collider component and collision hulls.  The link above has debug mode enabled so you can see the debug helpers in action as well.  You can see the blue outline of the convex hulls (which are automatically calculated from points of the shape).  It doesn't run super fast, but removing the "?debug=true" from the link will run it without the helpers and provide a speed increase.  This runs fastest in Chrome and Firefox 4, at the moment.  Also, you should know that asteroids don't start checking collisions until they've been "alive" for at least two seconds (it's not a bug).

What this also points out is that I need to get the work that I've been doing with dirty rectangles working.  The framerate is dependent on how quickly a browser can redraw the entire frame, which isn't the best way to do things.  I should be able to push a lot more collisions on that screen, but at the moment it's shaky.  Additionally, the particles are really slowing things down because there are so many of them.  This is just a good point when it comes to using all of the effects in the engine - don't do it unless you're okay with a low framerate.  The Asteroids Evolved demo is intended to be a demo, not an example of a full game... Maybe in the future it will run a lot better and I can polish into a real game, but for now it's a tech demo and nothing more.

Debug Helpers

When you're developing a game, it can be very helpful to "see" what's going on with your objects.  Thus, in v2.0 I've been merging the debug helpers into the components directly.  When you enable engine debugging with Engine.setDebugMode(true), or use the debug=true query parameter, it will enable all of these debug helpers.  What they are, are things like bounding boxes or collision hulls for collider components, the origin axis for transformations, and image outlines for billboards.

These kinds of helpers just make it easier, at least on me, to verify that something is occurring and that it's occurring correctly.  For example, last night I was trying to figure out why some changes I made to the Asteroids demo were causing the objects to spin about an origin that didn't seem to be where I thought it was.  Enabling the debugging helped me to see that the origin had been moved and correct the issue.  Using debug mode increases the frame render time, since it has to draw these helpers, but it's only while debugging is enabled.  All of the helpers are in pragma:DEBUG blocks, so they won't be compiled into the production version of the engine.

There have been some changes to some of the components, including the Vector2DComponent.  I noticed that render components were setting the bounding box for their host object.  Originally I intended this to be the way render components worked, but I've realized that while this is nice to have, it can also be painful to debug.  Plus, when I changed the way bounding boxes are created (they now have a top-left origin of 0,0 and only a width and height) this broke because most vector-drawn objects have their origin at the center.  Transforming the origin would typically put it at the lower-right corner of the object box, so it was breaking all across the board.  As such, a vector component will calculate a bounding box that you can retrieve from it (and now a convex hull) but it becomes a manual process, rather than an automatic one.

Also, I got the abstracted SoundSystem working last night with the Sound Manager 2 system plug-in.  It was a nice feeling being able to plug-in and remove the sound system entirely, and still have the game function regardless.  Now I will start, in earnest, making the HTML5 system plug-in.

I suspect that when people try to migrate from versions of the engine, prior to v2.0, they will be a bit shocked.  I've been noting these changes in the release notes for v2.0 mainly because there are a lot of little things like this.  It's all in an effort to make the engine faster, more optimized, and cleaner.  The difference between the v1.5.3 engine and the v2.0 engine will be fairly dramatic.  But overall, I think people will be happier with it...  It's really grown up a lot.

Debugging with Aptana

If you notice a reduced framerate and  you can't figure it out, try opening your debugger's "Breakpoints" tab and removing them all.  I've been hunting (for days) where the slowdown was occurring, profiling the engine and all of it's parts to no avail.  Finally, when I couldn't see my current breakpoints (valid ones) anymore, I removed them all... lo and behold!  The framerate issue disappeared.

Apparently Aptana still transmits all of the breakpoints, but just because they aren't valid doesn't mean that Firebug isn't checking them.  <sigh>  What a waste of my time - but a valuable lesson learned!

Profiling the Engine

Last night I did some research into what's eating up cycles when running a simple test (I used tutorial 11 from the v2.0 branch).  It appears that rendering is where the majority of the time is being spent.  I tested everything from Container to PooledObject, Iterator to SpatialGrid in the hopes of finding something that was really devouring time.  When I finally got to SpriteComponent, RenderContext, and CanvasContext, it came down to drawing paths and drawing sprites.  While most of the profiles were returning < 1ms average for every run, these areas (especially drawing images) were taking the most time.  Most profiles would execute 2000 times with a combined total of 300 or less milliseconds, while the actual drawing of images was doing 1800-2300 ms total for 2000 execution cycles.  While it isn't something I can directly improve upon, there are some things which can be done to mitigate the issue a bit.  But this brings up the question: is it worth spending the time on?

Dirty Rectangles

Recently I've been looking for a way to optimize rendering in the engine.  A lot of ideas are out there, but the best one (by far) is "dirty rectangles".  This is something that will apply to the canvas context since it's typically wiped for each frame and completely redrawn.  I'm just starting on this so I don't expect to have anything for a little bit.  But, if this results in increased framerates (like I hope it will) it'll push the engine to a place where people can expect more from it.  As it stands now, a game is completely limited by how quickly a browser can redraw the entire canvas.