Software Performance Engineering - Computer Systems Performance Evaluation - Lecture Slides, Slides of Computer Science

Download Software Performance Engineering - Computer Systems Performance Evaluation - Lecture Slides and more Slides Computer Science in PDF only on Docsity!

Performance

Engineering

Software Performance Engineering

What’s In This

Document?

An ACM “Queue” Podcast interview with a performance analyst.

Sample help wanted ads – what does the market define today when looking

for a Performance Engineer/Analyst.

One aspect of Performance Engineering – building performance into a

product.

Sample Want Ads

• Conduct reviews of application designs, business and functional requirements
• Implement test plans and cases based on technical specifications
• Design and execute automated and manual scripted test cases
• Document, maintain and monitor software problem reports
• Work with team members to resolve product performance issues
• Utilize multiple test tools to drive load and characterize system performance
• Execute tests and report performance/scalability test results

Skills/Requirements

4-6 years post-graduation experience in QA testing client and server applications

Demonstrated experience with MS SQL server databases

Experience with running UI automated test scripts. Familiarity with SilkTest preferred.

Exposure to multi-threading and network programming

Undergraduate degree from top-tier computer science/engineering university

Performance

Testing

Sample Want Ads

• Use home-grown and commercial tools to measure, analyze, and characterize performance, robustness, and scalability of the EdgeSuite Platform
• Serve as a technical point of escalation to operations and customer care
• Debug complex service issues: service incidents, complex customer setups, field trials, performance issues, and availability issues
• Enable specific capabilities to our operational networks that are outside the capabilities of our Operations group
• Work across all technical areas in the company to enable innovative new solutions that span multiple technologies and services, often to meet specific customer needs

Skills/Requirements

• Familiarity with data analysis
• Experience in network operation and monitoring
• Depth networking principles and implementation, including TCP/IP, UDP, DNS, HTTP, and SSL protocols a plus.
• Thorough understanding of distributed systems
• Experience with principles of software development and design

Performance

Debugging

Sample Want Ads

The Performance Engineer will provide technical leadership to the organization in the areas of software frameworks and architecture, infrastructure architecture, middleware architecture and UI architecture.

The Performance Engineer is expected to have versatile expertise in application performance (DB, middleware, UI, infrastructure). This engineer will collaborate with all teams within IT to implement an application performance measurement framework using end-to-end performance measurement and monitoring tools. Using data collected from these tools the Performance Engineer will work with the architects to influence application and infrastructure design.

This performance engineer must demonstrate skill versatility in the areas of application architecture, infrastructure architecture and application performance.

JOB RESPONSIBILITY

• Implement end-to-end performance measurement tools/frameworks.
• Build processes around tools to conduct application performance benchmarks.
• Design application benchmarks that will simulate application workloads.
• Design and implement capacity measurement tools and performance benchmarks and testing.
• Ability to wear many hats to help expedite multiple projects Skills/Requirements
• Strong performance measurement skills using tools like LoadRunner, SilkRunner.
• Strong performance analysis skills with a thorough understanding of application bottlenecks and infrastructure bottlenecks (OS, Storage etc).
• Strong skills using performance measurement/monitoring tools like BMC patrol, BMC Perform/Predict, HP OpenView, MOM.
• Hands on experience writing LoadRunner scripts and simulating performance benchmarks
• Experience with J2EE performance measurement tools is a plus

Performance

Architect

Docsity.com

Sample Want Ads

This individual will work with the systems architects and key stakeholders to develop a performance strategy for SSPG products and implement a methodology to measure fine grained resource utilization.

This individual will establish a set of benchmarks and a benchmark methodology that include all supported storage protocols, the control path, applications and solutions.

Will also be an evangelist for performance within the group and ensure that performance is a core SSPG competency.

The position requires strong “hands on” development skills and a desire to work in a fast paced collaborative environment. Candidate must have a strong knowledge of operating system technology, device drivers, multiprocessor systems, and contemporary software engineering principles.

Skills/Requirements BS in CS/CE plus 7-10 years experience or equivalent.

Proven experience with storage performance benchmarking and tuning including hands on experience with performance related applications such as Intel VTune, SpecFS, and IOMeter

Strong operating system knowledge base with a focus on Linux, Windows and embedded operating systems.

Strong C/C++ programming and Linux scripting experience.

Knowledge of any of the following protocols and technologies is a plus: iSCSI, TCP/IP, Fibre Channel, SAS, File systems, RAID and storage systems

Design and development experience with embedded system is desirable

Candidate should possess excellent verbal and written communications skills.

Performance

Architect

Performance Engineering

Motivation

A project is planned and scheduled under tight constraints; marketing feels that it is strategic to offer this product, and upper management inquires on a daily basis about the status of the project. Numerous short-cuts are taken in the design and implementation of the project. The product gets to alpha "on schedule", but it's discovered that the product is bug-ridden, and performs at 1/10th the speed of the slowest competitor. When the product finally ships, it's 6 months behind schedule, never wins a benchmark, and serves only as a line item in the product catalog. Within a year, a project is launched to build it "right".

• Does this sound familiar? Does this ever happen in your life?
• Is there ever a situation where such an occurrence is acceptable?
• In the above example, what if the quality was ok, but the performance remained terrible; would the scenario then be acceptable?
• Is it ever acceptable to not spec a product because there won't be enough time?
• Is it your job to put on blinders and do only what upper management tells you? Or do they implicitly expect you to develop it quickly AND do it right? In other words, are you "expected" to exercise a bit of insubordination?

Example: Read the unbiased, "true-to-life" example portrayed below and answer the questions posed.

Performance Engineering

Motivation

**1. Can you get performance for free? Does it naturally fall out of a "good" design?

2. Can you add performance at the end of a project?
3. Are performance problems as {easy | hard} to fix as functional bugs?
4. Is it easier to design in quality or performance?
5. It’s often stated that since performance is decided by algorithms rather than by coding** methodology, it's primarily project leaders or high level designers who need to worry about **performance. Do you agree with this?
6. What are the politics of performance estimation? What happens if you don't meet your** performance goal? What will happen if you up-front make your best guess and then your product comes in below this guess (after all, it was a guess, just like we've been doing in **class?) Does putting an uncertainty on the number make it OK?
7. Does it help to have management stress the importance of performance? When it comes to** the crunch, does management emphasize Performance, Quality, or Schedules?

LOTS OF OTHER QUESTIONS RELATE TO THIS TOPIC:

Performance Engineering

Motivation

MANY systems initially perform TERRIBLY

because they weren't well designed.

Problems are often due to fundamental

architectural or design factors rather than

inefficient code.

Performance engineering is no more expensive

than software engineering.

THE REALITY IS:

Docsity.com

Performance Engineering

Motivation

Good-performing systems result in:

• User satisfaction
• User Productivity
• Development staff productivity
• Selling more systems and getting a bigger paycheck.
• Performance can be "orders of magnitude" better with early, high level optimization.

THE BENEFITS OF PERFORMANCE ENGINEERING INCLUDE:

Timely implementation allows for:

• Staff effectiveness
• Fire prevention rather than fire

fighting.

• No surprises.

Performance Engineering

Introduction

In this section we begin looking at some of the practical ways of doing Performance Engineering. Performance Engineering isn't magic or miraculous, but an organized mechanism for building in performance.

Little’s Law ….. Utilization …. Blah, blah, !!!

Then a Miracle Happens!!

An amazingly good-performing product results.

YOU NEED TO BE A BIT MORE

EXPLICIT ABOUT SOME OF THE

DETAILS!!

KEY POINTS IN PERFORMANCE

ENGINEERING

By setting broad, verifiable performance targets

at the beginning, in the Marketing

Requirements, we can track those targets

through the whole development lifecycle and

verify along the way that the goals are being

met.

The goal is to show how to incorporate

performance information into the Standard

The trickle down philosophy:

KEY POINTS IN PERFORMANCE

ENGINEERING

Bootstrapping

FUNCTIONALITY PERFORMANCE

REQUIREMENTS Where the specified functionality fits in the market.

Where the specified performance fits in the market. FUNCTIONAL SPEC What it will do; interfaces. Resources needed by each of the functions. To what extent are functions used. DESIGN SPEC/TEST How it will/does work. Specific performance costs.

KEY POINTS IN PERFORMANCE

ENGINEERING

Performance Engineering depends on a combination of verification and validation. For those of you who've forgotten this nuance, here's a brief review:

VALIDATION AND VERIFICATION

Validation Showing at project completion that the performance meets

the stated goals.

Verification Showing at each stage in the development that the

projected performance will meet the previously stated

goals.

The costs to VALIDATE performance?

Establish performance goals.

Establish performance tests.

Schedule time for Performance Assurance to do their thing.

Schedule time to fix the performance.

The costs to VERIFY performance?

Establish performance goals.

Establish performance tests.

Schedule time for developers to conduct analysis and inspections.

Schedule time for Performance Assurance since no one will believe you've verified the performance.