The Engine Survey: General results

During the past few console generations, using a licensed game engine has gone from being a rarity to something common and acceptable. During the past few months, the number of middleware game engines available has suddenly increased from a relative few to something that needs a spreadsheet to keep track of. Whether the increasing number of available engines is due to developers deliberately addressing a perceived need in the market, or simply diversifying their revenue stream due to the lousy economy, is not immediately clear. So it seems like a good time to take a look at the game engines available and take the pulse of the game industry on game engines. What are we really looking for, and are there market opportunities for new game engine middleware entrants? In the early part of February 2009, I sent out a survey to industry executives asking for their feedback on the use of game engines.

Recently I’ve been speaking with many game engine middleware creators about the problems they are trying to address, their roadmap for the future, and how they are trying to distinguish themselves in the market. As an industry consultant, I’ve been doing this mostly to be informed and able to recommend one solution over another to my clients, but I am also just really curious – what are the problems that these game engine manufacturers see in the industry and are trying to solve, and are they the same problems I was trying to solve when working as VP Technology at a game publisher? Are we all on the same page… or not?
Regardless, I knew I needed more information, and any information I could gather from the industry would doubtless be useful to folks who are building game engines. I decided to solicit the feedback of the game industry by sending out several surveys – one focused on production, and one on technology –to appropriate industry executives. After a few weeks I had received 100 high-quality responses from industry members in senior management positions. This blog post is the beginning of a summary of these responses. Now, clearly I am not a professional survey designer, and I learned a thing or two about the art of creating surveys by screwing up a few questions. The information I gathered on these botched, over-generalized or otherwise useless questions has been discarded. The responses from the rest of the survey questions tell an interesting story, which I plan to detail here and over the course of several more blog posts.
This first entry will focus on general information learned about the industry’s perspective on game engine middleware. I’ll follow it up with posts focused on technology, and on production.
The Game Engines
Certainly there are quite a number of game engines one can license for a game. The ones that are relevant to any particular game vary based on targeted platforms and genre, as well as timeline and budget. A quite thorough list of game engines can be found on Wikipedia’s game engine page.
For the purpose of this survey I focused on these commercial engines, primarily in the “core” market:

• Crytek’s CryENGINE
• Digital Extremes’ Evolution Engine
• Emergent Game Technologies’ Gamebryo
• Epic Games’ Unreal Engine
• Garage Games’ Torque
• Id Software’s idTech
• Terminal Reality’s Infernal Engine
• Trinigy’s Vision
• Unity Technologies’ Unity
• Valve Corporation’s Source Engine
• Vicious Cycle Software’s Vicious Engine

There are many more engines available, including many open source engines. Some other notable engines not focused on in this survey include Blitz Games Studios’ BlitzTech, Digini’s Blade3D, and Dassault Systèmes’ 3DVIA Virtools.
Foundation of the Survey Responders
The approximately 100 executives whose data are used for this report are spread across the game industry, but by-and-large focus on the “core” market, PCs, consoles and handhelds, targeting traditional gamers. The casual and mobile markets have different engine demands which in some cases overlap with core, but the results of this survey should not be taken as directly applying to those markets. The core market appears potentially very fertile and lucrative at the moment when compared to others.
So for your reference throughout this data, our survey responders stated that they are currently working on titles for the following platforms:

Development on other platforms (PS2, WiiWare, Mac, Linux, iPhone, other mobile) fell below this rate.
Current development budgets for these titles fell largely into two lobes: under $4M, and over $16M. 42.2% of responders are working on titles costing less than $4 million, and 42.4% were over $16M.

Game Engine Awareness and Use
With so many game engines available now, which of these have game industry members actually heard of? Which have they used? Clearly one of the most important elements for game engine companies is just getting their product in front of the right people, so if executives aren’t even aware of their engine, they’ve got big issues. The game engines with the most awareness from our responders were not a huge surprise, and both the technologists and producers had roughly the same level of awareness on all the engines. These were the top five:

However, asked which of these game engines they’ve actually used, the picture becomes more interesting (I’m leaving off the lower parts of the data purely for brevity):

What is immediately noticeable in this data is the massive awareness and knowledge of Epic’s Unreal Engine. It is the most well-known, and more remarkably, a majority of our responders have used it!
After Unreal Engine, the data drops off a cliff fast – with Torque, Gamebryo, and Source all roughly equivalent as the second most frequently used game engines, with about one out of five people having used each. The distance between Unreal and the rest is interesting.
Even more interesting is the massive awareness of CryENGINE even though there aren’t many games on the market using it. Let’s see how perceptions of these game engines correlates with this.
Perceived Usefulness
Beyond simple awareness of these engines, how useful do the executives think they are for their current projects? Those who have used the engines before (or are using them now) should have a very good idea, as should those who have evaluated them, but these ratings probably mostly belie a simple perception of a game engine’s capabilities. I asked our responders to “opt out” if they didn’t know how valuable each engine actually would be for their projects, or rate them on a scale of one to five, if they did know:

Something that pops out of this data immediately when combined with the previous questions is the belief that CryENGINE is the second most useful game engine middleware available, even though it is also one of the least used!
Also notable is the sudden emergence of idTech and Infernal on this top five list. Does it reflect a belief that game engines made by companies that are using them for their own games must be the most “useful”? Or, keeping in mind that people are responding to how valuable these game engines would be for their current project, it could be that these engines have been seen used for games of similar genres, and on the same platforms. Certainly some of the other game engines pride themselves on being more broadly useful across a variety of genres. This data seems to suggest a belief that genre specialization is valuable. Or, maybe we’re all making a lot of the same types of games. 🙂 It’s hard to sort the complete answer out of this set of data, but it is interesting to see.
Using Game Engines
Backing up for a moment, why do we believe that people even want to use game engine middleware? If it is true that game engines are becoming more accepted by the industry, it should open up the market for more engine companies. Certainly the leading perception is that engine use is on the rise. But let’s check it out.
When asked if they are using game engine middleware on their project, our responders definitely indicate that engines are popular. 55% noted that they are currently using a licensed game engine on their title – that’s specifically an engine, not middleware libraries.
But when asked if they WANT to use game engines, the opposite results were returned:

Although the majority of our responders are using a middleware game engine, it appears that they would rather not! By this data, it appears that perhaps a thin game engine which adeptly integrates middleware libraries from other companies may find more acceptance in the marketplace than one which tries to do everything on its own.
The Economy
The difference in the percentage of people using game engines versus those who want to is certainly interesting. Could the high level of use have something to do with the economy, or with the increasing cost of development? I asked the producers whether they’ve found themselves with tighter budget constraints due to the economy. 78.6% responded yes. But in the comments many people pointed to tighter control over their budgets, as opposed to diminishing budgets.
Following up, what has been the impact of these rising development costs and a dwindling economy? What concerns have increased most for developers in recent years? The five most popular responses were:

Increasing development cycle length, increasing headcount, trying to create game designs that stand out from the pack… these are fairly consistent game developer challenges. But looking for rapid prototyping and rapid iteration tools? Here we find interesting new news. Rapid prototyping enables a developer to more quickly draft and test game concepts for fun in the early stages of a project, and also use the prototype to acquire funding. Rapid iteration gives one the ability to quickly try out many ideas during development, improving the game through frequent experimentation and fine-tuning.
If rapid prototyping and rapid iteration are weighing heavily on people’s minds, what are they using now? And how many studios have live preview on the target platform in their current content pipelines?
I asked, “what are you using for rapid prototyping?”

Several people noted that are using their previous engine versions to create prototypes for the next game. But what do new studios use? It looks like they probably create one-off C++ applications, sketch things out on paper, or use Flash or Lua. I had suspected that more developers would be using C#/XNA due to the ease of quickly knocking out quick test applications with it, but only 5% of the responders said they are using this for prototype development. (However, 76% of developers are using C#/XNA for tool building.)
If rapid iteration is also a growing concern in game development, how many developers currently have the ability to do live preview on the target platform for their content developers (artists and designers)? According to the results, 62.5% currently have this capability. Several responders noted that they preview on a PC version of their engine and that this is good enough for most work. Certainly using the actual target platform would be even more valuable though!
One useful tool for rapid iteration during development is the use of interpreted scripts; if the designer can write in a script language and immediately execute it on the target platform (as opposed to a long recompile or download), the iteration cycle can become quite fast, enabling many experiments and much more fine-tuning of gameplay ideas. But what script languages are most people using? (This query supported multiple replies.)

It seems that Lua is the hands-down favorite for scripting languages at the moment, although one developer wryly noted: “Lua is a failure – decided before I joined.” Lua is a solution that can work, but people tend to have a love/hate relationship with it. Using Lua on performance-constrained platforms can definitely be a challenge if you don’t understand the ins and outs of Lua’s memory usage.
Making a Decision on Engines
The last thing I wanted to know from both producers and technologists was: What factors do they use in deciding whether to license a game engine, and which one to choose?
The overwhelming favorites were of course cost and time. If we purchase a game engine, we want to spend less money and time than we would have creating the engine on our own. Otherwise why would we bother? (We’ll talk about that question more in the upcoming production-oriented post!) In third position to money and time came genre relevancy, or making sure that the game engine works properly for the type of game you are creating. This emphasizes the importance of doing a thorough evaluation of any game engine you’re looking at, and knowing if a particular engine was already used for a game technologically similar to what you are planning to create (a first-person shooter, for example.) Also highly important to the responders was support and thorough documentation, a good-quality content pipeline, and the ability to easily integrate the game engine into one’s existing technology as well as with other middleware libraries.
Licensing a game engine is an important decision. Clearly many studios are doing it, and you should do a thorough evaluation of each candidate game engine before settling on one. Unfortunately there isn’t enough public information about most engines, so seeing a game similar to your own, or knowing people on other teams who have used a particular game engine for a title can go a long way to helping you with a decision. As we all focus more on rapid prototyping, rapid iteration, and live preview in content tools, we hope that the game engine creators are listening and will continue providing us the excellent tools we need to create amazing games.
Thanks to everyone who filled out the survey – you know who you are. I’ll post more information on the production and technology feedback soon.