Engineering, by the numbers.

A new Enterprise Testing function for the modernization of a thirty-year-old system. About 60 people, partly consolidated from testing teams across the organization, moved to AI-assisted delivery.

The mandate

Ford Credit was modernizing a thirty-year-old system alongside new vendor platforms. Each program tested its own way. Enterprise Testing reported at SVP and VP level through three reorganizations.

What I did

I built Enterprise Testing as one function and consolidated testing teams from across the organization into it. Across three programs, I led about 60 people on an $8M to $10M annual budget. I organized the function around five areas: performance engineering, test data, the testing platform, a testing center of excellence (a group that sets standards and trains teams) and AI. I replaced program-by-program testing with one strategy, one governance model and one set of KPIs. The teams adopted them through enablement: I gave the teams training, standards, frameworks and platforms to use on their own, instead of owning each team's tests. I weighed build against buy. Four automation frameworks became one AI-assisted Playwright framework. My team built parallel test execution in house, with no added license cost, instead of onboarding the LambdaTest SaaS platform. I bought SaaS platforms where they covered gaps in the delivery cycle. I wrote the Playwright framework myself, in TypeScript with an AI coding assistant, to show the team how an architect writes code with AI. The code followed naming conventions, linting rules, quality gates, git pre-commit hooks and metadata-driven tests that take parameters. I put every team member on an LLM coding assistant. I designed the in-house AI agent, an orchestrator over six agents, and hired the engineers who built it. It read features from Jira, gathered context from Epics and Stories, and built the customer journey, then the test plans, test cases and automation: the complete functional testing lifecycle. I brought in PactFlow for contract testing, Tonic.ai for synthetic test data, HeadSpin for mobile testing and Parasoft for service virtualization. I negotiated the enterprise contracts for these SaaS platforms with procurement, finance and legal, within the planned capex and opex budget.

The result

• With AI, working out a feature's customer journey took 25% less time.
• The in-house AI agent cut the time to write a test by 66%, from about 60 minutes to about 20, and saved about 302 engineering hours in one quarter.
• Active users of the in-house AI agent grew from 8 to 38 across 7 teams in one quarter. The agent authored more than 450 test plans and test cases.
• The effort to supply test data for each account fell from between 1 and 2 days to between 3 and 4 hours, with a team built for it.

Quality engineering built from zero under HIPAA and GDPR. I led the merger of the company's AWS environment with that of its acquirer, Optum. All three managers I hired still work for its parent, UnitedHealth Group.

The mandate

A healthcare data platform under HIPAA and GDPR needed a quality engineering function. It also needed proof for compliance that no sensitive production data reached test environments. The company was acquired, and its AWS environment had to be merged with the acquirer's.

What I did

I led a team of about 20 people across the US, Latin America and Asia. I hired its three managers (performance and observability; test data; frameworks and contract testing). They helped me manage the rest of the team, who were contractors in Latin America and Asia. I set one platform-plus-standards model for the product teams, a shared platform with common standards, and built the capabilities to onboard them onto it. I introduced a cross-functional testing capability: performance testing with Grafana k6, contract testing for an API-driven platform, test data management with Delphix, and testing frameworks on Playwright for API and web UI. I masked protected health and personal data (PHI and PII) before it reached test environments. I made quality part of the release decision with CI/CD gates, run as GitHub Actions pipelines. For critical incidents, the gates could be bypassed through a defined process. I also brought in SaaS platforms to cover gaps in the delivery cycle: SonarQube for code quality, JFrog for build artifacts and container scanning (JFrog Xray), and Datadog for monitoring. After the acquisition, I led the merger of the two AWS environments. At the end, engineering worked under the acquirer's AWS policies. The merger took a transition plan across the organization, and testing and validation after the switchover window, with a rollback plan ready.

The result

• Compliance received validation reports as evidence that no sensitive production data had reached test environments.
• The AWS merger was completed, and engineering ended up working under the acquirer's AWS policies.
• All three managers I hired still work for UnitedHealth Group.

A B2B engineering organization built from zero for a fan-experience and ticketing platform.

The mandate

A unified fan experience and ticketing platform for professional sports teams and venues needed a B2B business. It also needed an engineering organization behind its revenue goals.

What I did

I started the B2B engineering organization from zero and staffed a founding team of 5 across the US and Latin America. I set the platform architecture (event-driven microservices on Kafka and Kinesis) and its standards, with templates so new services started from the same setup. Working one to one with Product, I translated three things into the technical roadmap: the revenue goals, a move to direct-to-consumer experiences and less dependence on third parties. With Product I created the core of the ticketing system: the data store, B2B back-office features, price tables and the tools to create live events such as games and concerts. I designed the backend and the event-driven flow end to end, using software events. A B2B back-office app publishes them. A core service decides how each software event is handled after the game or concert is created. The purchase app consumes those events. I also brought in AppDynamics for monitoring.

The result

• The B2B engineering team I started eventually took the platform to production.

Pipelines for Insights and AIQ, two data products for mortgage lenders, taken to production. Company network use cut from 80% to 40%. AWS spend on the AIQ pipelines down 60%.

The mandate

Insights is a benchmarking and market intelligence application for mortgage lenders, built on aggregated, de-identified data from Encompass, the company's mortgage loan platform. Its legacy .NET ingestion application could not hold the platform's load. A second product, AIQ, used artificial intelligence and machine learning to automate manual loan workflows for lenders, investors and servicers.

What I did

For Insights, I proposed, designed and led the replacement, an event-driven pipeline on AWS. API-driven microservices coordinated the Lambda functions. The pipeline could replay ingestion data from any stage of processing. The rebuild exposed a dependency returning full mortgage histories that averaged over 1 MB. For AIQ, a separate machine-learning product, I built the pipelines from client data intake to model inference on AWS SageMaker, working with the data science team.

The result

• When the platform went to production, company network bandwidth use rose to 80%. The company was negotiating to buy more bandwidth at the time. Compressing the mortgage-history payloads cut network use to 40%.
• AWS spend on the AIQ pipelines fell 60%.

A new backend, connected apps and streaming infrastructure went live together for the opening game. The backend was load-tested to 200,000 requests a second, and the platform held.

The mandate

NFL Media replaced everything at once for the opening game of the season: backend, apps for mobile, tablet and TV, streaming infrastructure, deployment pipelines and nfl.com. The team was new to microservices. Testing was a 25-person manual QA team.

What I did

I rebuilt the QA team around code. I kept 5 testers with engineering skills, hired 5 SDETs (engineers who write test code) and added 5 contractors for mobile test automation. My team took over all testing for NFL Media: standards, contract testing and observability across mobile, tablet, web, backend and data stores. My team defined and implemented the quality gates that governed the CI/CD pipelines. API, web UI and performance tests ran in parallel in CI, each able to block the release. Every pull request had a code-coverage gate. The integration environment had to stay green. Working with the deployment team, I co-designed one-click deploys that took a commit to production in about 20 minutes. My team owned contract testing across integrations, performance testing and the test frameworks. I appointed five technical leads and led the AppDynamics (application monitoring) rollout. I brought in BlazeMeter for performance testing and Sauce Labs for mobile and browser test automation. I set the release rules. Roll forward when production breaks: fix and redeploy instead of rolling back. Cut calls to third-party vendors that cannot keep up with load. Deploy under game-day policies, the rules for how code ships on a game day. In year two I took on Fantasy Football and worked with Product on new features and performance gains. I also owned the move from a vendor-managed data center to the NFL's own.

The result

• Everything went live together about 5 months after I joined.
• The backend was load-tested to 200,000 requests a second, twice the baseline from a test of the old platform.
• Working with the deployment team, I put the tests and quality gates in place, so a developer commit could reach production in one click, in about 20 minutes.
• Fantasy Football hosting fell from about $1M to about $100K a year.
• The QA team rebuilt around code had 60% fewer full-time testers.

As the Associated Press reported, 38,000 tickets sold out in 44 minutes. 7,000 buyers at a time, with more than 100,000 waiting.

The mandate

Major on-sales had been bringing ticketing platforms down. The Burning Man sale queued more than 100,000 people, far more than the platform could serve at once.

What I did

I led the performance preparation. I worked out the platform's limits. I put a virtual queue in front of the sale, on the festival's own domain, so 7,000 buyers could purchase at a time while everyone else waited.

The result

• The platform stopped going down on sale days afterward.