Yahoo Engineering Platform

1
Yahoo! Engineering Platform

• Ideas for a New Development Model

2
Whats Not Working?

• Poor development models
• Resource allocation
• Product flexibility

3
Poor Development Models

• Need technical product managers
• Core platform products rarely have a technical
  product manager. For example, UDB, message
  boards, alerts, auctions, classifieds
• Informal concepts of releases, feature list,
  product planning, documentation
• No formal process in place for feedback from
  users
• Service vs. Software
• First preference to offer as service not software
• No release standards/process in place
• Old style architecture requires complex
  infrastructure
• Service model requires centralized resources
• Business based technical solutions not
  centralized
• For example, auctions, classifieds

4
Resource Allocation

• Huge centralized teams
• My count 115 platform engineers supporting mix of
  products, platform services
• Reality is that even this is not enough. Platform
  products are often under resourced
• Permanent resources
• Business assigns resources based on revenue
• Frequently no domain knowledge of how the
  centralized service would apply in various
  vertical markets

5
Product Flexibility

• One size fits all approach
• Not everyone has heavy page view needs
• Designed with very limited flexibility
• Simple features are not added for years
• Inferior products
• Products built without domain knowledge
• Limited re-use
• Several different verticals developed several
  feed systems which often do the same task
• Businesses complain about lack of competitive
  features
• Service priorities are not properly judged
• Job alerts were not working for nearly 2 years
  before acquisition of hotjobs
• Common technology to cleanse boards of offenders
  never built
• Stifles innovation
• Frowned upon to have multiple solutions to same
  problem

6
Innovation Is the Solution

• Better Process, Better Methods, Better Answers.

7
New Development Model

• Inspired by the open source model
• Customized for yahoo!
• Leverage resources
• Encourage innovation
• Prepares yahoo! For long term success

8
Service Vs. Software

• Build software for customers
• Every service offered by y! Core platform should
  be installable
• Even if it is for in-house use, it should be
  structured like its meant for a customer
• For example, I should be able to download,
  install, and configure message board service on
  my machines with minimal effort
• Service operators use software
• Separation of service operators from developers
  usually brings out all the hidden issues
• Service operators can be international, or
  vertical specific with in US
• Document software for use

9
Development Model

• Sourceforge is the role model for tools needed
• Project starter webpage.
• Various roles for developers
• Source available
• Central help pages, message boards
• Easy to document product history, feature
  requests, bugs, etc.
• Technical Product Mgrs come from central teams
• BUs contribute resources as needed
• Developers join and leave the project group
• People can join in as a developer, project
  manager, product manager etc

10
Current Development Model
Product A
Product B
Service
Product C
11
New Model
Platform Product by Tech. Prod Mgr Architect Core
dev. Temporary Resources From BU
Product A
Product B
Product C
Product D
Platform Service by Service Operators
Product E
12
Resourcing

• Platform team is focused on
• Identifying product manager/administrator for
  core products
• Providing architects/core developers for key
  products
• Collecting requirements and preparing a product
  plan
• Business units can be confident of
• Understanding how resources breakdown between
  product/service/software
• Contribute temporary resources specifically aimed
  at improving platform
• Choice of solutions that is suitable for their
  problems
• Core products get resourced adequately

13
Innovation

• Choice of solutions
• Model for platformizing business solutions
• Flexibility to develop solutions for business
  needs
• Development model that scales across multiple
  locations and across countries

14
Recommendation

• Setup an internal source forge based site.
• Have developers submit projects
• Experiment with adoption