Current Features (March 2009)

Derange has been in development for quite some time now, so I figured it might be a good idea to review which features have been implemented so far to see where I’m currently standing. Many of these features have been rewritten from scratch at least a couple of times as I learned better ways to implement them.

Implemented features

• 3D graphics rendering using Ogre3D v1.6 (Shoggoth).
• Scriptable API using Lua v5.1 and LuaBind v0.7.
• Input device management using OIS v1.2.
• Support for multiple Locations with separate resource groups that can be initialized using DotScene files.
• Importing of DotScene files using OgreMax v1.6.23, creating necessary game objects as needed.
• Configuration through XML files using Xerces-C v3.0 for parsing and validating against XML schemas.
• Efficient resource management using Ogre3D’s capabilities.
• Flexible game object design, tightly bound to Ogre3D’s scene graph implementation.
• Multiple Controllers can be combined and attached to game objects to separate representation from the actual game logic applied.
• Easy to create game objects using a factory class which only requires the names of the object and the state to initialize it to.
• Skeletal animation support with smooth blending between different animations.
• Actors rotate smoothly in the direction they’re moving to using Slerp interpolation.
• Actors move using configurable speed and inertia parameters.
• Third person player control, with movement relative to the current camera’s position using the keyboard.

Near-middle future plans

• Collision detection, possibly physics simulation too. Need to figure out the pros and cons of the different third party physics engines available to decide.
• Automatic camera selection based on target proximity.
• Walking through predefined path nodes.
• Path finding.
• Mouse picking.
• Overlay manager for HUD display, menus, and on-screen debugging.

Resource subsystem – part two

In my previous post, I explained how the Resource subsystem interacted with resource managers. This time, I’ll show you in more detail how each Derange resource manager deals with their specific resources using Ogre’s resource management scheme. Again, I’ll try to stick to what’s relevant to the engine, because that is quite a lengthy topic.

Resource life cycle

A Resource goes through several state transitions until it is actually loaded. This is taken care of by its corresponding Resource manager silently. Here’s a chart with the different states and what they mean:

State	Declaration?	Instance?	Available?
Undefined	No	No	No
Declared	Yes	No	No
Unloaded	Yes	Yes	No
Loaded	Yes	Yes	Yes

1. Undefined: This is the initial state for all Resources. Basically means that the engine knows nothing about them yet.
2. Declared: The engine knows about the Resource, but no steps have been taken towards loading it, and there is no valid instance yet. Also, no memory is used at this stage.
3. Unloaded: Resources enter this state when the ResourceGroup they belong to is initialized. By this time, they are only using a small amount of memory: enough to hold an instance, but with no available content yet. They can also be forced back into this state by manually unloading them or when their ResourceManager exceeds its memory quota, which means the instance is kept, but not the actual content.
4. Loaded: This is the “fully active” state of a Resource. It means there’s a valid instance available and the data it contains has been loaded into memory, so it is available for the game to use. Resources are loaded automatically when they are needed, unless their ResourceManager exceeded its memory quota.

Class diagram showing Resource and Serializer hierarchies. (click to enlarge)

Serializers

Whenever a ResourceManager needs to load a Resource from disk, a Serializer is used. They are classes that implement the logic needed to read or write files of a certain type.

Typically, a ResourceManager is asked to load a resource  of a given name. Then it creates an instance of the appropriate Serializer and calls its load() method passing a DataStream with the file contents and a Resource instance as parameters. The file is then read and parsed in order to initialize the Resource with the content. And that’s pretty much all they do.

Parsing scripts

Some Resource types might depend on other Resources being loaded. Take Material scripts, for example. You might be loading many of them that use the same Texture, and it would be really sloppy to have it reloaded every time a Material requests it.

Instead of that, it is declared, which basically means that it will be loaded only once when the ResourceGroup containing the declaration is bulk loaded. This saves a considerable amount of loading time, depending on the times it is referenced.

Loading Scenes

Scenes are stored in DotScene format, as exported from 3DStudio Max by OgreMax. This is basically an XML file defining each object in the scene, their properties, and SceneNode relationships.

OgreMax also provides code to create the necessary Ogre objects when importing it into the engine, and a callback class that can be used to trap specific events generated by the process.

Derange traps some of these events. For example, if an Ogre Camera is imported, a suitable GameObject of type “Camera” will be created and attached to it as an UserDefinedObject. This saves the need to have two different scene graphs, since each GameObject can be reached from its Ogre counterpart.

As far as loading goes, the DotSceneSerializer only takes care of loading the .scene file and store the XML text contained into a DotScene resource. All importing is triggered from the resource itself.

Loading Game Objects

GameObject definitions are also XML files, but they contain several GameObjectStates that can be applied to an object, and are referenced by an unique name.

There’s only one GameObject class. It’s the GameObjectState that defines the type of GameObject: cameras, lights, actors, path nodes, etc. I will explain the details on these in another post.

This Resource, however, is a little different from the others: In order to load a GameObject, not only its name is required. we also need the name of the GameObjectState to apply. To achieve this, Ogre provides a way to define creation parameters per Resource through the StringInterface class. This is essentially a list of properties that can either be used by a Serializer as loading parameters (In this case, the name of the GameObjectState to load) or by an editor to display onscreen.

Resource subsystem - part one

Finally. I have been very busy researching and coding since my last post, but at last it seems to be paying off.

The subsystems were the first group of classes I developed for Derange (the skeleton, remember?), and as I tried different design approaches, they had to adapt to lots of structural changes, which ultimately led them in quite an amount of disarray (i.e. they were a total fucking mess). This made them perfect candidates for a complete makeover, so I took some time to redesign them to make them more flexible and hopefully, more elegant as well.

The one requiring the outmost care was clearly the ResourceSubsystem, because most other classes and even subsystems work with resources, and thus interact with it closely. So I decided to start there, and since right now I think it is much more robust and flexible, I will try to explain how it was designed and how it works.

The ResourceSubsystem is responsible for operations involving reading, modifying or writing files in disk, and performing them in an efficient way, reusing resources as needed or unloading them if they are no longer in use or if too much memory is in use.

Fortunately, Ogre already offers that functionality (and much more) and can be easily extended by implementing custom resource types. However, explaining in detail how it works is quite a big subject, so I'll try to stick with what is relevant to Derange only.

Ogre works with named groups of resources, using the interface provided by the ResourceGroupManager class. These groups may contain any type of resource that has been previously registered, and they can be stored in a file, compressed archive or even a remote URL. This is very useful as I can set up a default or "shared" resource group which contains all global resources, and then separate groups for each Location in the game, which can be loaded and unloaded as the player travels from one location to another.

Each resource type has its corresponding resource manager (which is derived from the ResourceManager class), and a resource serializer (derived from the Serializer class), which is simply the class that knows how to load and save a file of a specific type. If multiple resources come from say, a text file, you might be able to reuse the same serializer in all of them.

Regarding resources and resource managers, here's what the Ogre API says:

Resources are data objects that must be loaded and managed throughout an application. A resource might be a Mesh, a Texture, or any other piece of data - the key thing is that they must be identified by a name which is unique, must be loaded only once, must be managed efficiently in terms of retrieval, and they may also be unloadable to free memory up when they have not been used for a while and the memory budget is under stress.

A resource manager is responsible for managing a pool of resources of a particular type. It must index them, look them up, load and destroy them. It may also need to stay within a defined memory budget, and temporarily unload some resources if it needs to stay within this budget.

So, taking all this into account and after searching a long time for documentation on some very specific tasks (implementing resources with creation parameters seems to be a topic no one usually discusses -or I couldn't find anything on it-, so I had to take a look under the hood and figure it out myself), I came up with a new design. Here's a diagram that illustrates a part of it:

ResourceSubsystem class diagram. (click to enlarge)

As this class diagram shows, the ResourceSubsystem class keeps an instance of each resource manager class and two Log class instances: one for the system, and the other specifically for scripts. This helps debugging the engine and scripts separately. Additionally, it uses the CoreConfiguration class to access the engine's configuration, which is stored in an XML file and parsed as soon as the engine starts. The ResourceFileManager template class derives from Ogre's ResourceManager class and performs common operations like telling resources to load themselves and registering/unregistering the derived resource manager with the ResourceGroupManager class. All derived classes are supposed to implement only resource-specific operations. Additionally, all the classes in the diagram are Singletons, since there's no point in having multiple instances of them.

In practice, a simple usage example would be having a Lua script request that a specific GameObject is created and initialized to a certain state. These would be the steps involved in performing that:

1. The script asks the SceneSubsystem class to load the GameObject, and after this, the GameObjectManager will require a GameObjectState to be retrieved from disk.
   
2. The ResourceSubsystem::loadGameObjectState() method is called with the name of the GameObject wanted and the name of the state as parameters.
3. The load() method of the GameObjectState resource manager is called, passing the object name and a list of creation parameters. The state name is in this list.
4. The resource manager calls its base class ResourceManager::getByName() method passing the name as a parameter.
5. The resource is searched in all the registered resource locations (i.e. directories). Let's assume it was found and the resource is valid for this example's sake.
   
6. If the resource instance was not created yet (i.e. it was not used before), it is now, passing the creation parameters from step 3 along.
7. The GameObjectState::load() method is called.
8. The ResourceGroupManager::openResource() method is called, and a data stream is returned from the file.
9. A GameObjectSerializer class instance is created, and it is instructed to load() the data stream into the GameObjectState by parsing it and then using the setter methods on the former class.
10. That's it. The GameObjectState should now be loaded and ready to be passed back to the GameObjectManager that requested it in the first place. The next time someone requires that object with that state, it won't need to be loaded again. Unless it was unloaded to preserve memory, of course.

Right now there are only four resource types in the engine, but adding a new one is really easy. Just subclass the ResourceFileManager, ResourceFileSerializer and ResourceFile classes, add some method to handle those on the ResourceSubsystem class and voilá. There you go.

And this is how the resource subsystem interacts with resource managers. But there's still more, I haven't yet shown in detail how the manager-serializer-resource interactions work.

However, I'll be dealing with that on the next post. Hopefully sooner than what it took to write this one.