Tag Archives: Open Source

Open Source and Game Development

LBP_PSP_2_zpsf5435119

Game makers compete to make the best games. However, we don’t compete to make the best tools, whether they are content creation tools, pipeline tools, analysis tools, or testing tools.

We’d all reduce our development costs and make better games if we could share the cost of developing great tools. In turn, better games should attract more gamers and more gamer money, which would benefit all of us. We all lose when bad tools lead games to crash, stutter, or otherwise leave gamers disappointed.

Unreal, Unity, and others have excellent tool sets, but they can’t solve all our problems. If they did, we wouldn’t need to hire programmers.

What opportunities do we see for open source collaboration in the game industry?

To start, I know we all need to make tools. Professional tools can have a daunting list of features:

  • Cut, copy, paste
  • Undo, redo
  • Search and replace
  • Source Code Management (SCM) integration
  • Bug tracking integration
  • Ability to push updates
  • Crash reporting
  • Skinable
  • Keyboard shortcuts
  • Testable and Scriptable (say using Python)
  • Syntax highlighting
  • Spell checking
  • IntelliSense
  • XML import and export
  • Docking and window management
  • Standard UI controls
  • Session recording and playback
  • Multi-document editing
  • Support localization

Media or Game specific tools would add even more features:

  • Timeline controls (A lot of media has a time component)
  • Statecharts (like Visio) (A lot of media has states)
  • A variety of 2D, 3D or audio controls, meters and widgets (think Maya or Sound Forge)
  • High performance 2D and 3D viewing
  • Audio and video capture and playback. Variable rate playback.
  • Metadata extraction (All media has useful metadata)
  • Control via multi-touch (A single slider may not be ideal for media manipulation)
  • Layers (to allow multiple content specialists to work on the same ‘objects’ at the same time)
  • Separation of target and host (when you develop on PCs and target consoles or less capable devices, or multiple devices with varying formats)

Unlike throw-away tools, professional tools are hard to write (from scratch). Luckily, most of these professional features are common from tool to tool. Users want common, or better, identical functionality across their tools. They hate relearning variant UIs for each tool. Programmers adding new features to old tools also don’t want to have to learn a unique code base before they can be productive. The solution to making professional tools quickly and inexpensively is a high quality, modern tools framework.

Bill Budge came to Sony Worldwide studios nine years ago to create a tool to make tools, especially game tools. We called this effort ATF, for Authoring Tools Framework. We open sourced ATF on GitHub at GDC 2014. Although ATF offers most of the features listed above, this post isn’t a review or description of ATF. (For information on ATF: https://github.com/SonyWWS/ATF. Also, see ATF’s extensive wiki at https://github.com/SonyWWS/ATF/wiki)

We open sourced ATF, not only to do something nice for the game development community, but in the hope that ATF might be the start of something bigger. We want ATF to be a foundation for a long-term, open source effort enabling many game (and game platform) makers to work together on valuable (but non-proprietary) technology.

We aren’t the first company to open source tools or game technology. But game developers lack a place where open source contributions are vetted by a community of top developers. As Linux or LLVM have taught us, it’s not enough to upload some old tools or modules to GitHub and claim you are supporting open source.

Your contributions must be useful to many game developers. They need to pass code review, so they need to be well architected, documented, and include robust automated testing (so it’s easy for others to continue development). They should be modular, flexible, and efficient. They must pass all build or continuous integration tests. They should leverage common frameworks. Breaking changes should be minimized (and work arounds documented). New components must work with existing components and be non-duplicative. Like any ongoing software project, refactoring will be needed on occasion.

We believe game companies can collaborate if we focus on non-proprietary pieces, the stuff your top programmers don’t want to do, the boring stuff, if you’d prefer. Stuff like:

  • Automated testing
  • Performance Analysis
  • Common content editing tools (level editor, script editor, state machine editor, etc.)
  • Common networking components
  • Common social components
  • Common business components
  • Common pipeline components
  • Common asset management, asset workflow, and asset analysis
  • Analytics and Metrics

We want ATF to be an enabler for this long-term effort. If you’re interested, we hope you join our community, https://github.com/SonyWWS/ATF, and start contributing!

What to tell your boss about Open Source?

If your company is spending significant resources on proprietary software that―

  1. isn’t your company’s competitive secret sauce,
  2. would be interesting and useful to many programmers, and
  3. doesn’t duplicate other open source technologies,

―you should either commercialize it or open source it. You’re wasting your company’s money if you don’t.

This is good place to note that commercial software can also be open source―that these terms are not mutually exclusive. You can open some or most of your company’s source and require a paid license for commercial use (even if academic or non-profit use is free). You can keep just enough proprietary (even for a relatively short period of time) so that anyone who can afford it will pay for the whole thing, not just the open parts. You don’t even need to decide exactly what to do up front, just start with the safe (to open source parts) and see how it goes. That’s what we’re doing with ATF. 

Next Post: Open Source and Game Development

What costs are associated with Open Source?

Assuming you have one of those rare projects that may attract free help from other companies or individuals, what costs are associated with making it Open Source?

  • Legal: Every line of code should be vetted for proper licensing and being free of contractual, trade secret, patent and governmental restrictions.
  • Refactoring: You’ll need to cleanly separate your code into open source and proprietary repositories.
  • Start up costs: You’ll need to jump-start the open source community. Some minimal marketing, PR, and sales costs are likely before you get to critical mass.
  • Documentation: It will be easier to get adopters if they don’t have to read the source tree to figure out if it might be valuable to them. Existing documentation may need to be expanded, edited, or ported to a wiki.
  • Automated testing: As you add more developers, your need for automated testing increases, so you may need to add or enhance tests.
  • Oversight: Until you can recruit trusted open source partners, all outside check-ins must be vetted for bugs, utility, and foolishness.

These costs are small, but if no one joins your community, you’ll waste your company’s money.

For more on starting and running an open source project I recommend:

Next Post: What to tell your boss about Open Source? 

What should be Open Sourced?

What projects should companies open? Choosing which project to open source is just like any other work decision. Prioritize the work with the highest projected ROI. For open source, these are the projects that will attract the largest community (of free helpers, or more specifically, free programmers) without threatening your revenue.

Projects that appeal to programmers are the only ones that will recruit programmers. Things like:

  • Incredibly useful programmer stuff (TCP, email, HTTP, LDAP, Unix, etc.)
  • Projects that help programmers program (compilers, frameworks, IDEs)
  • Projects that minimize annoying tasks generally assigned to programmers (continuous integration, automated testing, setting up and monitoring machines, networks, environments, etc.)
  • Projects that give programmers access to tools and systems otherwise unaffordable (large databases, expensive instrumentation, etc.)

Just like commercial products, open source projects are competitive. If there is already a dominant, similar open source technology, unless yours is clearly better, you’re probably out of luck. Giving it away for free isn’t going to help, though a truly open license (e.g. FreeBSD) can beat GPL (because GPL is poison to commercial entities).

We open sourced ATF because it is incredibly useful to tools programmers, it helps programmers program, it minimizes annoying tasks, and it brings a variety of otherwise unaffordable bits into reach.  Making long-term success more likely, it has no significant open source (or even commercial) competition. If you want a tool to make tools, ATF might be your only choice. 

Next post: What costs are associated with Open Source? 

What should not be Open Sourced?

no-opensource

Google or Facebook would not be global leaders without the global standards of HTTP, Email, Hyperlinks, Unicode, JavaScript, HTML, and many other technologies. After all these successes it should be obvious that being secret or proprietary isn’t always smart. Only certain kinds of projects should be kept proprietary:

  • Your secret sauce. The magic elements that would reduce your competitive advantage by more than the savings and reach of open community development.
  • Information that, if disclosed, would cause more harm than the good open sourcing brings. Some source code contains trade secrets or information under NDA that cannot be disclosed. Happily, these typically can be excluded from open source distribution by simple refactoring, at some additional cost. Other source code may open the doors to a possible copyright or patent lawsuit or PR gaffes. (Oh my, the things some programmers type into the comment fields!) You’ll need to review every line of code, which unless you already do code reviews, has a cost . (And the benefit of a long overdue code review!) Additional refactoring may be necessary to minimize potential legal entanglements.
  • Lesser value projects that aren’t worth the trouble of converting to open source. No one wants to waste resources or reduce their ROI by staying proprietary unnecessarily. Unfortunately, very little of your (non-commercial) software has any value to anyone else. If you put it on GitHub as open source, no one would show up to offer free help.  Few enterprises bother opening their source code because they have nothing (non-commercial) worth open sourcing.

Thankfully ATF had been code reviewed, copyright reviewed, patent reviewed, and refactored. It wasn’t secret sauce. It did have substantial value to other tools programmers (and it doesn’t have any obvious open source competitors). Happily it immediately perked interest (#1 on reddit upon release) and our community is growing day by day. 

Next Post: What should be Open Sourced?

Why open source software?

logos_fullColor-atf

In honor of my group at Sony open sourcing ATF, the tool to make tools, and this week, the ATF Level Editor, I’m going to explain why open source makes sense for both individuals and companies (and especially game tool makers now that we can share a tool making framework, ATF) in a series of blog posts. I’ll start at the beginning.

Why open source software?

There are three kinds of software:

  1. Commercial
  2. Proprietary
  3. Open source

Many programmers work on commercial software, software made by one company and sold to everyone else. Its ROI is easy to measure: simply add up revenues and divide by costs. For instance, game developers make commercial software. However, not all software developed by game teams is sold. They also develop tools of all sorts, which are rarely sold. These tools are an example of proprietary software.

Most programmers develop proprietary software, software used within a single company. Proprietary software is built to reduce internal costs, not for resale. Unfortunately these cost savings are hard to calculate. Without a clear return on investment (ROI), businesses often treat proprietary software as a pure cost, best minimized, which might be penny-wise but pound-foolish. (To find the value of such software, an investment in analytics is required.)

A few of us work on open source software, software made by either individuals or companies and given away for free to anyone who accepts the license. Linux, Firefox, Chromium, much of Android, and parts of Apple’s OS X are well-known examples of open source software.

Why do individuals participate in developing open source software?

1)   Some programmers are passionate about some kinds of software and so work on these open source projects for fun and enlightenment. This is the same reason people play music, run, or paint. Also, like music or running, this gives people a chance to be part of a group of people who share their interests. We take great pride in being part of a tribe.

2)   Individuals can master their chosen specialty by working on open source projects. Their contributions are open to the world, displayed to any company interested in acquiring the skills demonstrated.

Why do companies participate?

  • Razors and razor blades: We give away something up front to unlock future revenues. Phone companies give away phones (but not service contracts), razor companies (almost) give away razors (but not razor blades), printer companies discount printers (but not toner), game publishers give away ‘free’ games (but not micro-transactions), and companies that sell Linux services give resources to support Linux. They give away the hard to monetize ‘standard’ parts like HTML, JavaScript, or Linux, and sell high value services on top, such as Google ads or professional services.
  • Divide and conquer: Companies are banned from coordinating pricing, labor costs, or ‘unfair’ divisions, even when these would benefit the consumer. However, collaborating on open source software is legal and saves money. Instead of ten companies each building a proprietary compiler (for instance), ten companies can split the development cost for a ten to one savings.
  • Two heads are better than one: When companies’ experts work together, the result is better than if they worked apart. (When Sony shut down its proprietary tool chain efforts and joined the LLVM consortium, it ended up with a better tool chain and saved money.)
  • Trust, but verify: No company wants buggy software. To find more flaws, open software to more eyes. No programmer wants black boxes, as APIs and documentation may lie. Only source can be trusted. When something breaks, source is needed to find the problem. All software has flaws, and unless you don’t care about them, you want source.
  • Build community: A successful open source project will attract partners, potential employees, and efficient global PR, marketing, business development, training and support.  If you need a global standard to enable your business, choose open source.

Next post, what shouldn’t be open sourced?