Title: Grids in EDA Software Development
1Grids in EDA Software Development
gridMatrix Technology
Tom Grotton Cadence Design Systems,
Inc. Director, IT Server Farm Initiative
2Corporate Needs
- Need more computing capacity
- Products are becoming more complex and taking
additional cycles to build and test. - Products need to be supported on multiple
platforms (OS and version) - Process and machine utilization measurements are
inadequate to effectively plan and manage capital
equipment requirements. We continue to add more
and more servers to meet this unplanned demand. - Need a more reliable hardware, storage and
network - Nightly runs of product builds and tests fail too
often due to machine or infrastructure problems - To decrease the dependency on individual hardware
resources - Need more cost-effective IT contribution
- System administration costs are ever increasing
as more and more hardware is added. According to
Gartner and based on our experience, TCO costs
are being primarily driven by support costs and
not capital. For every 20 cents spent on
equipment, 80 cents is spent supporting it.
3How We Address these Needs
- Manage Cost (Total Cost of Ownership)
- Agility (Adaptive IT infrastructure)
- Increase Quality (Product and Service)
- Virtualization (Computing, Networking, Storage,
Security, File systems, Monitoring, and Common
Process Architecture) - Federation (Brokering, Provisioning, Resource
Sharing, Globalization) - Automation (Process, Workflow and Ease-of-use)
4 Capitalizing On Our Expertise
- The key point of coordination, information
exchange and collaboration for those involved in
large-scale grid projects in the US, Europe,
Canada and Asia-Pacific is the Global Grid Forum
(GGF). GGFs platinum and silver sponsors are
Compaq, HP, IBM, Sun, Microsoft, Platform
Computing, AVAKI and Entropia. - As a member of the GGF, Cadence chairs and leads
the - GGF Common Use Case Model Architecture Working
Group, formerly known as NPi - This Working Group is responsible for defining a
particular lightweight, high-level architecture
for distributed computing management. Creating an
overall Reference Model for advanced distributed
computing management - GGF Job Submission Description Language working
group - This Working Group is to define a standard Job
Submission Description Language (JSDL) for
describing a computational 'job' and its required
execution environment for submission to a Grid.
This language would be used to construct a
document that would encapsulate all of the
information needed by a Distributed Resource
Management (DRM) system or a job submission
system on a Grid, such as a Grid scheduler, to
place a job in its required environment for
execution. - GGF Grid Policy Architecture working group
- This Working Group is developing the requirements
and architecture for interoperable policy
management, and for how they are described,
evaluated, stored, managed, distributed, and
enforced. Since policies may also be associated
with instrumentation to provide feedback into
evaluating the effectiveness of the policies, the
requirements and architecture will also provide a
framework for associating the appropriate metrics
and instrumentation with the policies. - Is a seated member of the GGF DRMAA (Distributed
Resource Management Application Architecture)
working group. - Reference The 451 Group Grids 2004
5Grids
- Definition Grid computing refers to a network
architecture designed for large-scale dynamic
sharing computing resources. A grid works by
taking the responsibility for input/output
requests for storage, memory, processing, and/or
communications resources away from individual
machines and instead moves that responsibility to
a network grid that searches for available
resources to handle resource requests. - Usage A growing number of companies are
considering deploying Grid Technology or have
deployed it. These companies are finding the
Total-Cost-of-Ownership to deployment, management
and support Grid Technology to be overly costly. - EDA is the slowest industry to adopt massive
parallel processing grids. - Partly because of the industry itself and partly
because of the nature of the products. - Industries that are using Grid Technology are
Life Science, Engineering, Commerce, Financial,
Electronic Design Automation, Digital content
creation, Testing, Server/Storage, Batch
processing, Decision support/data mining.
6Grid Industry
- 2005 grid Hardware and Software solutions 18
billion - 2005 grid Software solutions 4 billion
- There are several middleware DRM (Distributed
Resource Management) companies Platform, Sun,
IBM, etc. - There is an increase in next-generation DRM
products that are no-cost. - There are also products that have their DRM
layer, GridIron, DataSynapse. - Enter Utility Computing - With the emergence of
this universal computing grid, there will be a
need for a new breed of utility provider.
Although companies and individuals will retain
some local computing capabilities, there will be
a growing opportunity for trusted neutral parties
to operate and manage shared resources within the
grid. (ASPnews 9/02) - Projected growth of the software grid technology
market is 25 until 2007, then it jumps up
considerably due to the larger number of
industries that will start to use grid
technology. EDA is one of these.
Data sources http//www.gridpartners.com,
http//www.idc.com, http//www.econstrat.org/,
http//gridcomputingplanet.com,
http//www.bloor-research.com/,
http//www.gridforum.org/
7Grid Technology Evolution
- Compute Grids share excess PC compute cycles to
provide a high-performance, low-cost computing
environment. Their application is scientific
research and engineering. - Example, SETI_at_Home.
- Market leaders are Entropia, Sun, Platform, and
United Devices. - The EDA industry deploys this kind of grid, but
in a UNIX environment and not across the
internet. - Information Grids are distributed architecture
designed for large-scale dynamic sharing of
commercial and technical applications, data, and
compute power within the enterprise or across
multiple external organizations. They are used
in specialize industries, such as,
pharmaceutical, biotech, medical, financial
services, oil and gas exploration. - Example, IBM/Upenn Mammography research project.
- Market leaders are AVAKI, Platform, IBM, Sun, and
HP.
Data source The Grid Report, Bloor Research,
November 2002
8Grid Technology Evolution
- Service Grids make use of the intersection of
grid and web services technology concepts. As
such, the service grid provides the underlying
architecture for Utility Computing model. Sun
describes a Service Grid as collections of
services brought together from across the
network. Service Grid are composed of services
that you need at a particular time mail, stock
feeds, word-processing, or flight information,
etc. The idea of the service grid is that each
piece comes from a source that specializes in an
external function. - Examples, Sun ONE and HPs Utility Data Center
OGSA. - Market leaders are HP, IBM and Sun.
- Intelligent Grids are self-managing utility
architectures that go beyond the bounds of
department and enterprise massive extended
enterprises encompassing partners, suppliers,
and customers. Currently, intelligent grids use
homogeneous resources. - Examples of intelligent grids are IBMs eLiza
IBM Workload Manager, HPs Planetary Computing
project, and Suns Management Center, N1. - Market leaders are IBM, Sun, and HP.
Data source The Grid Report, Bloor Research,
November 2002
9Grid Technology What to do?
- Utility Grids are to some extent conceptual to
the grid industry. They use the power utility
model, where there are main generation grids and
peak request or overflow local grids. Utility
grids takes the position that computing resources
and the DRM layer are commodities, because these
resources can be provisioned on demand. Utility
grids are capable of managing higher-level
functions, such as, security, data access and
transformation in a massively parallel
environment across the enterprise. - According to several industry sources, Compute,
Information, and Service grids will likely
converge into Intelligent grids in the future.
The fifth grid type Utility Grids or Corporate
Utility Computing Grids are evolving and will
likely replace the Service Grid and Intelligent
Grid in time. This grid is designed to make
heterogeneous computing power and heterogeneous
resources available on-demand.
10What does the future look like? Utility Grids
11Cadence Grid
- Inside Cadence we are deploying Grid Technology
today! So far, it has resulted in an average of
67 reduction in process job run-time for RD and
Services business units worldwide. - We have around 3000 CPUs in our grid. Next year
we are looking at doubling this! - RD and Services business units require IT
provide grid control, monitoring, statusing,
error/alert reporting, performance reporting and
job optimizing for massively parallel processes
across hundreds and thousands of heterogeneous
computer resources, ranging from Microsoft
Windows, Linux and UNIX (AIX, HP-UX, Solaris). - Cadence has developed and deployed patented grid
technology. - As most of you have done or are trying to do, we
have developed an abstraction layer that performs
Grid Automated Process Control allowing a
Developer, CM, PV, Flow, Support engineer to
interface with complex job definitions in a
natural manner. Today, LSF is the Distributed
Resource Management (DRM) layer and networked to
this are Data Grids, which make computer
resources, networking and storage virtually
invisible. - Results - Significant reduction of
Total-Cost-of-Ownership by reducing cost of
adoption and deployment, job run-time, costs in
support, capital equipment, personnel and the
technical knowledge.
12Yesterdays Process
Harness Scripts (fire up DRM - bsub)
Logs Email Reporting (Some Web Reporting)
Run Job or Tool
Hardware (Servers/Workstations)
13Todays Process
This is the only interface that is seen!
Engineer submits a grid-enabled job
Intelligent Job Controlincluding web-based
monitoring and job control
DRM
Hardware (Servers/Workstations)
Resource Monitoring Alert Handling(DRM, system
and system configuration)
14What is Needed
Standard means to grid-enable EDA Toolby thread
Engineer submits a grid-enabled job
Intelligent Job Controlincluding web-based
monitoring and job control
DRM
Hardware (Servers/Workstations)
Resource Monitoring Alert Handling(DRM, system
and system configuration)
15 Where are we?
Intelligent Grid Technology
Dependency Management
Site Grid Health
Site
Workspace Management
Heterogeneous Resources
Heterogeneous Process
Natural Language
Compute Grid Health(Commission Platform)
Local
Job Definition(Process Control)
Process
Multi-Site/Cluster
Job Control
Job Priorities
Job Execution
Platform Definition
Utility Grid
Workspace Management(RCS, ClearCase, CVS, VNC,
Extendable)
Alert Handling(GRS)
Dynamic Web Reporting(Customizable)
Self-Healing
Process Metrics
Job Statistics
Metrics
Job Monitoring
Queue Definition Optimization
Host Groups and Host Partitions
Workspaces
Pool Configuration
DRM Grid
Job Distribution
Res. Control
Commission and Monitoring
16Site Grid Status
17Horizons
Cycle-times reduced 67
Examples - 4 Hrs to 17 Mins - 13 Hrs to 2 Hrs -
6 Hrs to 30 Mins- 24 Hrs to 3.5 Hrs
- Corporate Grid (2005)
- Multiple site grids sharing resources and data
intelligently across the WAN. - gt50 utilization
Site Grid (2003) Combines compute farms to
facilitate the sharing of capital resources
across multiple departments, projects, and
groups Increase server utilization from gt30
Compute Farms (2002) Enables shared system usage
across departmentsIncrease Utilization from gt11
18QA
19(No Transcript)