3 Real-World Stories of PE Portcos Struggling (and Succeeding) that are not about AI

It’s almost impossible to open LinkedIn, social media, or your kid’s PTA newsletter without reading about how AI will change the world. There is no question that inside of our industry it’s having what is in many ways an unprecedented effect compared to other technologies.

Even private equity backed CEOs and CTOs are getting pressure from the firms and the board to adopt AI and use it for software development, product enhancements, to cut costs, etc. Some of these things can happen easily, and some organizations are still working through the challenges of past decisions like M&A activity, and moving an on-prem product to the cloud.

These are the kinds of companies I work with everyday. I’m even helping these companies adopt AI! But I’m here to remind you, for every company you hear featured on a private equity podcast, or taking the stage at some industry event, there are plenty of other companies that are doing the hard work of getting better every day as well.

In the DevOps movement we used to go to conferences and hear about the latest amazing accomplishment from Amazon, Netflix, and Google. However, one of the best of these talks I ever attended was the Google engineer who talked about the Java program, wrapped in a Python program, wrapped in a Golang program. Yes, even the vaunted Google sometimes uses duct tape and baling wire to keep some project moving.

To that end, when you’re feeling overwhelmed by the AI pressure, remember, you are not alone. Below are three stories of companies just like yours, that struggled, and are still working on getting better, every day.

We’re Never Going Back

Situation description: The company had been given a mandate to move out of the data center and into the cloud. They would get a better multiple on exit, just for being in the cloud. So they hired some “experts” to do it for them and they shut down most of the data centers.

Nothing else really changed. They still used the same tools, they deployed software the same way. Only now, when they needed new servers, they used the Azure console, instead of driving to the data center. Mostly. There was still one data center remaining to be the single point of failure.

Outcome: Of course, because someone else had moved everything for them, nobody really knew how anything was set up, and nobody really knew how things worked in the cloud.

How to fix it: It’s not that the engineers were not smart or capable. This is how the company had always solved their problems. Hire someone else to fix it.

The first thing I explained was that they were going to have to rebuild everything. From scratch. Unless they understood how everything was created, and could do it themselves at any time, there was no way they were going to be able to effectively operate that amount of infrastructure.

We even did a white board exercise where they drew their entire deployment pipeline and architecture and I went through piece by piece, crossing things out and diagramming how it would be handled in a cloud-aware way.

The engineers started writing Terraform, and Ansible. They could build and destroy whatever they needed. Environments started to look consistent from Development all the way to Production. Engineers drew maps of how software got from a developer’s desktop to production. And they started making improvements.

It will take them years to get to where they want to be, but they have the knowledge and the capability to do it.

When asked how they felt about the way they used to do things compared to now, the engineers said: “We’re never going back”.

You’re Not Facebook

Situation description: Like many PE backed companies, they had originally been on-premises for lots of clients. They had moved to the “hosted” stage where the instances had been migrated to the cloud and now they had thousands of hosted instances.

Because each instance had been, and still was, separate, there were customizations that had been made for each customer on each instance. There was even a team of technical account managers who were specialists in making these customizations.

Outcome: Over time, different groups had developed their own tools for making customizations, and for managing the instances. In many cases, there were two or three different ways to do the same thing.

One of the places that was customized was the database schema. Because they were able to take advantage of the services their cloud provider had to offer, they proudly told us that they had over 1000 database instances, one for each customer.

In the early days of Facebook, it was known for running thousands of instances of MySQL, before they started building their own databases. We asked “You’re running 1000 databases and you’re not Facebook?”

They were confused at first.

How to fix it:

We gave a number of specific recommendations in the Service Delivery Assessment.

• Deployment is performed on-demand by developers or operators without the need for multiple engineers or steps.
• Chat and/or revision control is used to give visibility and repeatability to operational processes.
• The deployment process for software and infrastructure is well understood and documented.

There were a few more, but these highlight some of the challenges that were getting in the way of this company progressing from the hosted stage to multi-tennant SaaS. It’s really difficult to allow for customizations, to rationalize database schemas, and to leverage scalable infrastructure if it’s difficult to ship and no one knows what is happening.

We’ve checked back with the company over the years after having done dozens more assessments. At first, there was a lot of cultural resistance from leadership to making the necessary changes. Now they are in a much better place.

Time is Money

Situation description:

“What if we invest in training our people and they leave? What if we don’t and they stay?” - manager’s wisdom

One of the side effects of doing a lot of M&A is that there can be lots of different ways to do things. When building a platform from scratch, good teams will follow consistent principles so that once an engineer “gets the hang” of a platform, they will have reasonable guesses about the right way to get something done.

When a company is a combination of many different things, all developed independently, and there has never been a serious effort to rationalize the platforms and processes…it can be a lot. Depending on the specific things you are trying to do, it will be necessary to track down the person with the right answer. It’s almost impossible to find the answers in the documentation. But learning who is the right person from which to extract the tribal knowledge is a challenge in itself.

Outcome: When things are different and it’s very difficult to come up to speed on how things are done, onboarding can take a long time. Typically it should take about 2 months for a senior engineer, and 3 months for a junior, before they can be expected to make meaningful contributions to an organization. For this particular client, it would take between 6 and 8 months, and sometimes longer.

That meant that there were many cases where the company was paying the engineer for an additional half a year of training before they could meaningfully contribute.

Yes, at large companies like Google and Facebook, there is extensive training that lasts for months. But Facebook was also known at one time for creating enough safety that engineers would commit code to production on their first day.

This can only happen with a lot of intention.

How to fix it: Ambiguity and tribal knowledge are not good for engineers. The DORA research on AI found that companies that had published an Acceptable Use Policy for AI had many times more AI adoption than places where it was ambiguous. Engineers like to understand the rules of the game so they can have the freedom to be creative within those constraints.

This also holds true for changes to the way things are done within a company. If we are going to start a platform engineering initiative, it’s important to be intentional about the way that platform will be adopted, how we will collect feedback, and how we will ensure that engineers are comfortable before they are required to rely on it.

If we can reduce the number of ways things are done, so that the parameters of the operating environment are well understood, we can get the maximum amount of benefit or any type of onboarding engineer in the shortest amount of time.

You are not alone

The process of improvement is a journey. Everyone is on their own part of that journey and it is a journey without end. In the DevOps movement we study Toyota and they are never satisfied that they’ve achieved perfection.

We say that “best practice is current practice” because the practice can always get better. Yes, there are many companies taking advantage of AI today. But also, there are many companies that are, or were, in the same place that you may be now.

You are not alone.