about me

Dave with angry eyes.

Howdy there! I've been making web things for over a decade and loving every minute of it. I'm particularly fond of bringing games and the web together in order to delight and enlighten players.

I'm a full stack developer in the open source web world - I tend to spend most of my time programming or ensuring software will launch and succeed (automating everything, coordinating people, counting every millisecond). In addition, I'm a talented team lead who strives to act as a force multiplier to make my teammates happier, more productive, and more effective.

work and projects

Last Updated: Feb 2014


Raidbots

Role: Everything
Team Size: 1
Launched: 2011
Status: Live
Core Tech:
  • PHP
  • Mongo
  • Varnish
  • JavaScript
  • jQuery
  • AJAX
  • Apache
  • Linux
  • git
  • CloudFlare
  • HTML/CSS
Noteworthy:
  • 100-200k visits/month
  • 6+ million characters
  • 200+ million character fights logged

Raidbots is a series of tubes for WoW raiders that I've built over the last couple of years. It initially started as a way to collect data about player performance to get at real-world metrics. DPSBot aggregates hundreds of thousands of boss fight combat logs to see a data-backed comparison of classes and specs. EpeenBot highlights a specific player's parses against her own class/spec to see where she lies in the greater scheme of things. CompareBot is a utility to dig into multiple boss fight parses in order to see small variations of ability rotations or gear disparities to understand how their own performance can be improved.

I've spent a lot of time and effort optimizing the page response time on Raidbots. Raw data is available from World of Logs once a day which I download and process as a batch. The processing denormalizes a lot of data to make database lookups as fast as possible. Varnish is used to cache pages until the next batch is processed. Raidbots uses CloudFlare as a CDN to both get assets closer to the visitors as well as for minification of CSS, JS, and HTML.

Raidbots is hosted on its own dedicated non-virtual hardware both to minimize contention and to manage its sizable database (the Mongo database sits at about 200GB right now).


Infinite Crisis

Role: Lead Developer, Operations
Team Size: 15
Launched: 2013
Status: Live
Core Tech:
  • NodeJS
  • CoffeeScript
  • MongoDB
  • Kue
  • Restify
  • Chef
  • Capistrano
  • MySQL
  • Nginx
  • Varnish
  • Redis
  • statsd
  • Graphite
  • PHP
  • New Relic
  • Pingdom
  • Systems Administration
  • Operations Architecture
Noteworthy:
  • Sharded MongoDB
  • Edge and Local Caching
  • PCI Compliance
  • Multiple Datacenters
  • Database Replication

Infinite Crisis is a MOBA set in the DC Multiverse. The web team manages several applications from the main website used for acquisition and engagement, the game launcher, and the in-game storefront.

Another operations engineer and I were responsible for all production aspects of the sites: build system management, deployments, server provisioning in multiple datacenters, PCI systems compliance, monitoring, and infrastructure design.

In addition to operational work, I led my team to design and implement a core game data integration system . This system is used to power applications such as referrals and a public-facing data API. I built the foundation for this system from scratch and managed all operational aspects.

Major accomplishments:

  • Multi-datacenter, sharded MongoDB deployment for game data integration system.
  • In-game store deployed to two datacenters to support three regions (NA, EU, RU). Flexible automation ready to handle future regions or datacenters as needed.
  • Used automated deployment tools to quickly push new builds without any interruption to public traffic.
  • Designed "Dry Dock" process to speed up a previously difficult workflow for pushing new code into PCI-compliant production environment. My process was later used by several other departments facing similar difficulties.


Turbine Web Infrastructure

Role: Operations, Developer, Architect
Team Size: 3
Launched: 2013
Status: Ongoing
Core Tech:
  • Chef
  • Capistrano
  • Vagrant
  • Pingdom
  • New Relic
  • PagerDuty
  • statds
  • graphite
  • Bamboo
  • nginx
  • varnish
  • git/GitHub
Noteworthy:
  • Faster, Safer, Repeatable Deployments
  • Dev Responsibility for Production
  • Proactive Production Mindset
  • Vast Reduction of Manual Processes
  • Infrastructure Overhaul
  • PCI Compliance

Before 2013, almost all aspects of production operations were manual: building production-ready artifacts, deploying code, managing servers, inspecting logs, and more. Deployments were rare, risky, and time intensive and dev/test/production environments were inconsistent. Minor fixes could take days or weeks to deliver to production.

Two operations engineers and I improved every area of this process using a number of tools and training developers to use them.

Configuration Management

We spent about a month working with Puppet to automate server configuration and application deployment. After running into a number of obstacles, we switched to Chef and have utilized it to consistently provision servers with identical configurations. Using community cookbooks, business-specific wrappers, and custom code, we can now bring a vanilla machine into any state we need: database server, application server, load balancers, and more. Chef is used in both test and production environments to ensure consistency throughout the application lifecycle. Manual provisioning is a thing of the past.

Continuous Integration and Delivery

I deployed and created build pipelines using Bamboo for all existing and upcoming projects. Commits to source control automatically trigger a build which that runs static analysis tools, executes unit tests, and create an artifact which can be shipped to a test or production environment. Bamboo also controls deployments to test environment that any developer can trigger at any time by pushing a button - these deployments run any integration or functional tests available.

Simplified Deployments using Capistrano

I built and used community Capistrano recipes to make complex multi-server deployments easily run from the command line as well as perform any ad hoc system management that hadn't yet been automated. Capistrano allowed us to easily deploy to multiple servers in parallel as well as coordinating different types of servers when needed - for example, we often deploy new code to a single server not handling public traffic for a quick smoke test before rolling it out. With one Capistrano command we could pull the server from the public pool and deploy the code. Once validated, a second command would put it back into the pool and then methodically perform the same sequence for all other production servers.

Operational Awareness

We brought in many new services and technologies to ensure that production was properly monitored and that critical problems would alert someone responsible within minutes of it being detected. Pingdom was used to monitor basic up/down status of sites, New Relic makes response time and errors visible, PagerDuty manages on-call schedules and alerting, and several open source projects like statsd, collectd, and graphite are used to inspect current and historical system state. The combination of these technologies have enabled developers to manage most production applications without requiring a deep background in server administration. The net effect is that we can deploy more quickly with fewer problems and react to new issues incredibly fast.

And More

Many other improvements were made during this process.

  • Migrated to Nginx from F5 load balancers to reduce cost and improve automation.
  • Implemented consistent backup schedules and tools.
  • Isolated sites from one another to improve security and reduce contention.
  • Utilized Splunk to aggregate all logs and provide insight to developers.
  • Created tools to assist developers with complex Perforce workspaces.
  • Deployed Hubot and wrote custom scripts to inform the team of production events (and pug bombs).

Image below not related to me or Turbine. Just awesome.

Cartoon Universe

Role: Tech Lead, PHP Developer, Operations
Team Size: 10
Launched: 2012
Status: Closed 2013
Core Tech:
  • PHP
  • MySQL
  • Apache
  • Linux
  • Akamai
  • Unity
  • Zend Framework
  • Drupal
  • JavaScript
  • jQuery
  • HTML/CSS
  • Perforce
Noteworthy:
  • Multi-region support
  • Akamai
  • Drupal/Zend Framework integration
  • Unity Client integration

I acted as the development (and often project) lead for the website for Cartoon Universe, a Light Persistent World developed by the Montreal WB Games Studio. I acted as technical contact with Montreal to integrate the Unity client seamlessly into our site. Working closely with our Web Operations group, we utilized Akamai to aggressively cache the site and game assets as well as handle geolocated access to the game. I also worked with our core web technology team to build our first implementation of our Drupal and Zend Framework integration (Drupal handling most CMS responsibilities, ZF handling more critical aspects such as authentication with WBID, account management, and web-based store functionality).


Gotham City Impostors

Role: Tech Lead, PHP Developer, Operations
Team Size: 10
Launched: 2011
Status: Live
Core Tech:
  • PHP
  • MySQL
  • Apache
  • Linux
  • Akamai
  • Hydra
  • Game data integration
  • JavaScript
  • jQuery
  • AJAX
  • HTML/CSS
  • Perforce
Noteworthy:
  • Leaderboards
  • Player stats and VS mode
  • Match views
  • Web administration of in-game challenge content

Gotham City Impostors is a multi-platform arcade style online shooter. The GCI website acts as the marketing and community center for the game with a number of game data integration features including leaderboards, player stats, and a snazzy match viewer.

I was a developer for the game announce site (game information, forums) and was the project lead for the game launch site (game data integration). I wrote much of the PHP backend that communicates with Hydra (a game data platform created by Agora Games) and presents game data to visitors. In addition, I worked closely with another developer to create the administrative UI to create and manage in-game challenges by the game community team.


W.E.L.P.S.

Role: Everything
Team Size: 1
Launched: 2011
Status: Live, Private
Core Tech:
  • Node
  • Client-side JavaScript
  • jQuery
  • AJAX
  • HTML/CSS
  • MySQL
Noteworthy:
  • JavaScript client and server side
  • Integration with existing Ruby app
  • Real-time Fleet Monitoring
  • Uses EVE's in-game browser tech

Real-time intelligence tool for EVE fleet commanders. Integrated with a custom Ruby web application to gather and broadcast fleet status to all corp members. Uses the fact that EVE's in-game browser sends player-specific information to a trusted server via headers on every HTTP request. The client-side of the application triggers an AJAX call to the Node server several times per second to update the server as well as receive any relevant state changes from other pilots. I built the application to integrate directly into our corporation's custom wormhole tracking tool. WELPS piggybacks on Ruby's session information and database.


Batman Arkham City

Role: PHP Developer, Operations
Team Size: 10
Launched: 2011
Status: Live
Core Tech:
  • PHP
  • MySQL
  • Apache
  • Linux
  • Akamai
  • Hydra
  • Game data integration
  • JavaScript
  • jQuery
  • AJAX
  • HTML/CSS
  • Perforce
Noteworthy:
  • Information Architecture
  • Joomla Customization
  • Logging and Performance Reporting Framework
  • Hydra Feed Admin
  • Screenshot and Video Galleries

Batman Arkham City is the sequel to the very successful Batman Arkham Asylum from RockSteady Games. The Turbine web team was tagged to create the marketing and game launch sites to promote the game, act as community hub, and integrate with the game itself (though this never launched due to external factors).

I was primarily a backend developer for this site with some front end implementation work as well. We heavily customized Joomla to work with our own framework to handle single sign-on and shared design. I worked on many of the informational aspects of the site including the news and galleries. In addition, I co-created a web UI to manage in-game messages of the day with another developer that was used to populate localized messages in the game itself via Hydra (game data platform created by Agora Games). This feed management system was also used for Gotham City Impostors.

I assisted with a number of other game data integration features that were never launched including a faction takeover system that would have changed the theme of the site based on in-game activity.


MyLOTRO / MyDDO

Role: PHP/Java Developer, Operations
Team Size: 5
Launched: 2008 (MyLOTRO), 2009 (MyDDO)
Status: Closed 2013
Core Tech:
  • PHP
  • Java
  • MySQL
  • Apache
  • Linux
  • Wordpress
  • Game data integration
  • JavaScript
  • jQuery
  • AJAX
  • HTML/CSS
  • Perforce
Noteworthy:
  • Deep WordPress Customization
  • Game Driven Accounts
  • Single Sign On
  • Player/Character Blogs
  • Friend Activity Feeds
  • Leaderboards

The LOTRO and DDO social networks were major initiatives at Turbine to increase player investment in the games, provide additional community tools, and establish a platform to build additional tools in the future. Initial features included player and character profiles, blog posting, event history, friend connections and messaging.

I was one of the developers for the integration of multiple systems and completely in charge of the WordPress implementation. Libraries that I created include the PHP code to understand and process the game data, friend activity feeds, tools for frontend data displays, and more. I worked on enhancing all aspects of the sites for both LOTRO and DDO over several major releases over two years.

I worked closely with another developer on the Java application that acted as the intermediary between the game data systems and the web. Together we created the underlying framework and schema for handling hundreds of thousands of character events per day and making that data available on the web.


LOTRO / DDO Character Signatures

Role: PHP Developer
Team Size: 1
Launched: 2008
Status: Closed 2013
Core Tech:
  • PHP
  • MySQL
  • Apache
  • Linux
  • GD
  • XML
  • Perforce
Noteworthy:
  • Flexible Image Rendering Pipeline
  • XML Templates

Character signatures for LOTRO and DDO was a project to connect game data to the web via a generated image for use in the Turbine forums and elsewhere on the web. I built the entire image generation system as my first project when I started at Turbine. It's been a popular feature that is still being used today.

The signature generator uses XML templates to generate an image using the GD library. The XML defines various image generation actions that build a PNG layer by layer. Actions available include placing text on the page, layering transparent PNGs based on variables, and more. Information about a character are available to use as variables in the template to do things such as using different portraits based on race and gender, showing attributes such as name and level, and so on.

The XML templates and isolated assets make it incredibly easy to update without requiring any code changes. New classes, races, or level brackets can generally be added simply as an asset drop without any downtime required.


LOTRO / DDO Data API

Role: PHP Developer
Team Size: 1
Launched: 2009
Status: Closed 2013
Core Tech:
  • PHP
  • MySQL
  • Apache
  • Linux
  • XML
  • Perforce
Noteworthy:
  • Developer API Keys
  • XML Output
  • Created for Player Devs

The LOTRO and DDO API applications were built to expose game data to players in a simple form so they could build their own applications or integrate with their own sites.

I built the first version of data.lotro.com in a few days as a skunkworks project to prove that the site would be fast to develop. It included a basic developer key system, 24 hour caching, and access to key character information. Over time we've expanded it to include XML data about items and guilds as well.

The system has been successful with players creating iPhone apps, website integrations, and libraries for various languages.


EVE Item Comparison

Role: Everything
Team Size: 1
Launched: 2007
Status: Live, Out of Date
Core Tech:
  • PHP
  • JavaScript
  • AJAX
  • Apache
  • Linux
  • MySQL
  • HTML/CSS
Noteworthy:
  • Dynamic UI
  • Price medians

EVE Online has some of the most complex systems of any MMO out there. Tools both in-game and out-of-game are rarely useful in displaying information in a way that is easily digestable. I built the Item Comparison Tool in order to quickly understand item differences and hierarchies.

I used JavaScript and AJAX extensively to create a client-side application that would let visitors search and select items, show only their differences, and sort them based on any attribute. In addition, I built a number of backend tools to update pricing information on a daily basis to be able to have up-to-date measurements on cost/benefit for each item.

While it's no longer online, I also built a webapp to understand the total worth of your own assets retrieved from the EVE API and using the same pricing systems I had built for the Item Comparison Tool.


Reason CMS

Role: Creator, PHP Developer
Team Size: 6
Launched: 2002
Status: Live
Core Tech:
  • PHP
  • MySQL
  • Apache
  • Linux
  • LDAP
  • HTML/CSS
  • JavaScript
Noteworthy:
  • Built for Higher Education
  • Flexible Data Types
  • Arbitrary Data Relationships
  • Powerful Form System
  • LDAP Integration
  • Used for all Carleton sites
  • Open source

As a student web developer, I was directed to create a general PHP-based system to handle content management on the college's News and Athletics site. This simple request begat a fully featured custom CMS that is being used for nearly all aspects of Carleton's website today as well as many other school's in the Midwest.

I wrote all the initial code to handle the very flexible database schema, admin UI, and basic frontend capabilities. Another year as a full-time developer on staff saw much more functionality and a migration to have most sections of the Carleton website using the CMS. Over the past decade, it's grown into an open source project used by a handful of colleges in the Midwest.


Kamigami - WoW Guild

Role: Guild Leader, Raid Leader, Raider
Team Size: 50
Launched: 2008
Status: Closed 2010
Core Tech:
  • Being a Bear
Noteworthy:
  • Friendly and Effective Guild
  • US Rank ~500 25 man Tier 12
  • Long-term friendships and relationships

While not technical, Kamigami stands out as an important project that I am still proud of to this day. My wife and I started out in Kamigami as normal members but quickly became core members and eventually ended up with full ownership of the guild as the original players fell away from the game.

Kamigami started out as any other random guild tackling 25 man raids. Over the course of two years, we managed to improve until we were roughly in the top 5-10% of all 25 man guilds. Our greatest achievement, however, was that we were able to create and maintain a friendly, adult atmosphere the entire time that avoided most of the stereotypes of other successful guilds (immature, unfriendly to women, negative/hostile atmosphere, etc).

Leading a guild taught me a number of lessons that I use every day: how to lead and work with a team, providing positive and negative feedback, making choices about recruitment, diffusing and resolving conflicts, and keeping morale up while working on hard problems.


Et Cetera

Here's a big list of smaller things and less interesting jobs I've done in no particular order.

  • CommandStar, a real-time communication interface on the web for Starbound. Written in CoffeeScript, powered by sockets.io to provide real-time relaying of game chat and information to the web, IRC, and HipChat.
  • WoW TCG Deck Analysis written in CoffeeScript
  • Installed, configured, managed Bamboo, Jira, FishEye+Crucible, and Confluence at Turbine. Trained most of the web team how and why to use these tools.
  • Built several continuous integration/delivery systems on top of Jenkins and Bamboo to manage application packaging, unit/functional/Selenium testing, code coverage, static analysis, and stable environment management.
  • Built a Twitter trivia bot that accepted guesses via DM and awarded the first N correct answers with LOTRO/DDO point codes.
  • Created several command line tools to help Turbine web developers handle complicated Perforce client specs as well as smooth over Windows/Linux complications for the less command line-y developers.
  • Recorded, edited, and published dozens of EVE and WoW videos using FRAPS and Vegas Studio (one of them has even gotten 30k views on YouTube).
  • Managed dozens of developer and QA engineer Linux VMs to maintain consistent environments for everyone.
  • Am familiar enough with Akamai metadata to be able to find and correct problems created by Akamai's own professional services...
  • Written Java webcode to manage gift cards in a warehouse using WindowsCE barcode scanning devices using an XML service that communicated with ancient PICK databases. Seriously ancient. Some of the queries/programs had a last modified date of 1984. This was in 2006.
  • Written Java to manage health care documents flowing through an Optical Character Recognition system that was validated by overseas data analysts
  • Built a web-based Instant Runoff Voting system for my college's student elections.
  • Took over 10,000 photos in college and built my own management system to share with friends before sites like Flickr had caught on.
  • Given several presentation on current and upcoming technology for my coworkers, including CoffeeScript, how to learn the command line, and more.
  • Built a killboard scraping system for EVE University to preemptively collect intel about enemy and pirate corporations to understand hostile tactics, routines, and capabilities.
  • Turbine Starcraft 2 Champion

I've toyed with all kinds of other technology not specifically listed above. This site only includes projects that I had a considerable hand in, made it live, and had some impact.

If you'll forgive me for keyword stuffing, I have been closely involved with the likes of: Python, Ruby, CoffeeScript, Express, Backbone.js, Redis, MixPanel, BlueHornet, Service-Oriented Architecture, SOAP, Scrum/Kanban development methodologies (including acting as a Scrum Master for several long term projects), RESTful APIs, TDD/BDD approaches and libraries such as PHPUnit/Mocha/Jasmine, lighttpd, nginx, HAProxy, Pound, LISP, Scheme, ML, and Prolog!

OK, the last few were from programming language and AI classes in college. But they were fun and I can't pass up a chance to sneak Prolog into my portfolio.


contact / around the web

  • email Email Me
  • LinkedIn Logo LinkedIn
  • github logo github
  • youtube logo YouTube
  • Google Talk logo Google Talk: [email protected]

Credit where credit is due

I'm not a designer! Thanks to these open projects for letting me make something that looks decent!





:wq