As a hosting service provider, here at Tibus we’ve seen an incredible pace of change in the application development world in the last couple of years. What was once a DevOps philosophy is now a body of practice, and the while DevOps movement has gained considerable momentum within out client base.
The benefits of a DevOps approach are now clear but, to be honest, we’ve only reached that point having first been proven wrong. We saw big risks in a very agile approach to software development. But experience now tells us those risks were overstated and that the improvements in speed to deployment and, more importantly, in getting software that delivers more closely to the initial vision of that software far outweigh the problems of loosened (or perhaps we should say, old-fashioned) change control.
DevOps is alive and maturing quickly in the enterprise sector, but how does it differ when the production environment is the web? Can DevOps goodness be enjoyed even when the production environment is now managed by an in-house ops team (either on-site or in a public cloud), but by a third-party hosting provider?
The questions above are the ones we have been answering over the past 14 months through our work with a number of organisations, including a large client from the banking sector. Following a change in technical leadership and a demand for faster-paced deployment practice, they asked us to support their move to DevOps and agile development.
We had to embrace the change that came to us and, to be honest, it was really hard. Here’s what we learned along the way.
The bread and butter of any hosting provider is solidity. No dramas, no problems, nothing to report - that’s not a boring day, that’s a day to high-five colleagues on the way out of the office, a day of professional and spiritual fulfilment.
Agile and rapid frequent deployment is, by it’s nature, laced with change and when a hosting company hears ‘change’, they see potential brain-ache developing toward the end of their working day.
Our clients have helped us along this path and we’ve learned to see the world as they do, but it really was hard to adapt our worldview to theirs at first.
The lesson here is to be patient with your hosts and providing they are willing to adapt, support them as they do.
A web host’s typical communication toolbox will be based around email and a service desk, probably combined with a service request portal or ticketing system. And, of course, a telephone number.
We’ve embraced Slack and now communicate via that. We rather quaintly use Skype (some dev types prefer the messaging format to Slack) and we use internet relay chat. We’ve changed our processes and have embraced the idea that developers want to use messaging and collaboration tools rather than email.
Change request forms and email chains just don’t work as a web host when you have become the ‘Ops’ bit of DevOps. We’ve got more work to do here, but we’ve learned that we must think of ourselves as the production team of a development effort. That means using the same communication tools as our clients do. Insist that your hosting company uses the collaboration tools that suit your own development culture.
In total candour, we realised sometime early in 2014 that we just didn’t seem to be talking the same language as our developer clients. It was a deeply unsettling realisation. We’d missed the genesis of DevOps and agile or, at least, we just didn’t get how fast it was taking root.
In a client meeting this became clear when they explained what they wanted to do, described the changes they were embracing and talked about what they needed from us. Our core values - robust platform engineering, good networking / IP peering and a bunch of clever technical chaps - just wasn’t going to be enough any longer.
We simply wouldn’t have understood this if it weren’t for sitting down with clients and listening. Talk face–to-face with your hosting company and explain what is needed for the new world - that’s what we needed to hear at the time.
Our hosting estate is governed by ISO 27001, and the platforms of our clients in the government sector are governed even more robustly. Most hosting companies are in the same boat.
That naturally leads us to think in terms of ring-fencing: everything together in a neat, controlled environment in-keeping with the criteria by which we’re governed. When a client developer asks us to change something, we’re immediately thinking: “Okay, what’the potential effect on our 3,199 other clients?”
The idea of a development-production continuum requires the production environment ring-fence to be lifted a little, or else expanded to include the development environment.
These are conceptual changes for hosting providers, but we’ve accepted they are a must for agile, rapid and/or frequent deployments. Listen to your hosting provider on what good change control is, but push to have the agreed change control demarcation point meet your needs.
Agile and DevOps is all about constant feedback, constant dialogue, constant decision points. We used to pride ourselves on making lovely server performance metrics available transparently. But an increasingly frustrated client showed us we needed to do much more in order to help him understand how their rapid development deployments were performing.
We’re using New Relic and AppDynamics to help these guys maintain the pace they want. As another example, we’ve embraced browser devkit outputs. Much of this was new to us and, again, challenged our notion of what makes a valuable host in a client’s eyes. We’ve changed our self-view from server folks to application performance folks. But we couldn’t have got to this place without a client explaining what they really needed from us in order for them to embrace DevOps.
Tell your hosting company what you need and ask them to give you the associated Ops-type tools you want.
It sounds obvious but the development, staging, UAT, pre-production and production environments should be the same. Even the sandbox. This is obvious to enterprise Ops people, but it isn’t obvious to a hosting provider. Encourage your host to establish and fully embrace a workflow that matches your development style and technical stack. We had to change our ways here: no longer could we expect development teams to take the responsibility for moving an application or release along that workflow, solving problems as they did. Our clients have asked us to be a part of their team and delve far deeper into ‘app world’ than we would have done previously. That’s been good and it’s been interesting – but we wouldn’t have gone in this direction until our clients told us explicitly this is what they expected from us, as their Ops people.
Tell your hosting provider to make that workflow meet your needs and get them to understand the underlying applications as much as you do.
We’ve come to realise in the last 18 months that we had to get closer to our development clients’ teams. Our professional, procedural, but slightly fussy way of working wasn’t suitable any longer.
We now have specialists for technical stacks (for example Drupal, native PHP, Wordpress, .Net and many other), where before we steadfastly refused to have vertical support practices at all.
We have ‘named support’ contracts - again a huge change in our processes - where clients work with individuals exclusively, and this has been a breakthrough. We were previously wrong about this because it is clearly working really well. It extends to more frequent, shorter, face-to-face meetings and even semi-social meetings with our clients.
We genuinely have to think of ourselves not as server guys but as the Ops guys for agile development efforts, an extension of the client-side developer team. Push your hosting company to consider a new closer way of working based on the DevOps ethos.
That’s DevOps from our specialist hosting service provider perspective. We’ve had to make changes and, to be honest, those changes weren’t invented here, but were mandated and driven by our clients.
The rate of change has been quite something over the last few years and that isn’t likely to slow any time soon. The DevOps and agile philosophies are here to stay - and that’s a good thing. Us hosting folks must adapt to them too, and that we are doing.
