Why data empowerment is key to effective cloud operations

Most organ­i­sa­tions are on a jour­ney to update their IT infra­struc­ture secure­ly and sus­tain­ably. Dig­i­tal trans­for­ma­tion is an almost uni­ver­sal busi­ness strat­e­gy and, at its core, it is about mod­erni­sa­tion. An urgency to trans­form tech­nol­o­gy usu­al­ly means some­thing is not as it should be with­in the organ­i­sa­tion, whether that’s feel­ing the rise of com­pet­i­tive pres­sures, the bur­den of lega­cy sys­tems hold­ing the com­pa­ny back or a need to be more effi­cient and pro­duc­tive.

There are many pos­si­ble cat­a­lysts for change but, no mat­ter what, a busi­ness need will be in the driver’s seat. Cloud tech­nol­o­gy is one such pil­lar of dig­i­tal trans­for­ma­tion where organ­i­sa­tions have increas­ing­ly realised the poten­tial val­ue and ramped up invest­ment. Even those reluc­tant to embrace cloud tech­nol­o­gy will have like­ly been com­pelled to change tack to nav­i­gate the dis­rup­tion of the pan­dem­ic. And in 2022, at least 74% of organ­i­sa­tions have a hybrid cloud strat­e­gy.

---

---

But a plan that stops at cloud migra­tion, or doesn’t take into account exact­ly how an organ­i­sa­tion will need to use its cloud infra­struc­ture (or might need to in the future) is des­tined for prob­lems. A cloud strat­e­gy is a con­tin­u­al jour­ney, not a des­ti­na­tion, and doesn’t end once an organ­i­sa­tion has trans­ferred work­loads to a new sys­tem. A strat­e­gy should be mapped out from care­ful plan­ning and prepa­ra­tion to exe­cu­tion and oper­a­tion.

Often this is less an over­sight and more a lack of resource and inef­fec­tive bud­get allo­ca­tion. One of the key issues organ­i­sa­tions come up against on their cloud jour­ney is hav­ing the right skills and exper­tise. Michael O’Don­nell, Ph.D, senior ana­lyst at Quest, explains “[many organ­i­sa­tions] can’t get the skills, peo­ple and resources to deliv­er the val­ue promised by cloud. For exam­ple, a big chal­lenge right now is that com­pa­nies may have a desire to shift to a cloud-native microser­vices archi­tec­ture. How­ev­er, the skills short­age means organ­i­sa­tions are still vir­tu­al­is­ing work­loads and mov­ing them to cloud ser­vices.” With this ‘lift and shift’ approach, organ­i­sa­tions are miss­ing a real oppor­tu­ni­ty for change.

“By exten­sion, the skills short­age is in itself dri­ving the need for automa­tion, and again the need for cyber­se­cu­ri­ty pro­fes­sion­als as more and more devices come onto the edge, e.g. con­nect­ed cars and the rise in IoT devices,” says O’Don­nell. “That’s repli­cat­ed across the board, in every role that a com­pa­ny might need for re-archi­tect­ing or their re-plat­form­ing/­cloud migra­tion pro­gramme.”

---

---

To tack­le this issue, organ­i­sa­tions need to work with a clear frame­work and roadmap, pri­ori­tis­ing what they can do today with the resources they have and what they need to work towards. “When you look at a data mesh, or data fab­ric or data con­trol plane, that’s a roadmap they can work towards,” adds O’Don­nell. For exam­ple, organ­i­sa­tions can work out what tasks to auto­mate, and pri­ori­tise ensur­ing that data is inter­op­er­a­ble so oth­er appli­ca­tions can use it. If you have a roadmap, you can break the pro­gramme into man­age­able chunks and build it up. To quote the famous proverb: “how do you eat an ele­phant? One bite at a time.”

Batch adop­tion has the advan­tage of break­ing down what can be an over­whelm­ing project, both finan­cial­ly and in terms of skill resourc­ing, into more man­age­able pieces over time. “It’s an effec­tive way to build on a company’s tac­it knowl­edge of cloud ser­vices,” says O’Donnell.

How­ev­er, some­times the costs for com­mer­cial-off-the-shelf (COTS) on-prem soft­ware may actu­al­ly end up small­er than the stack up of microser­vices. It would be impos­si­ble to know this unless organ­i­sa­tions have done some upfront enter­prise archi­tec­ture analy­sis. The busi­ness out­comes need to be in bal­ance with the desire of IT to push tech­nol­o­gy bound­aries and oper­a­tional effi­cien­cy gains.

A key chal­lenge is cloud oper­a­tional costs. The cloud mar­ket is grow­ing and spend is increas­ing, but many organ­i­sa­tions are strug­gling to man­age their cloud bud­gets. A recent sur­vey from Cast AI found that organ­i­sa­tions are over­spend­ing three­fold on cloud costs. The main rea­son for this is over­spend­ing on expen­sive resources, with lit­tle to no return. “It’s real­ly easy to over-allo­cate IT ser­vices with­in the cloud, with scope creep and all these ‘dark cloud’ ser­vices that get spun up like mush­rooms,” says O’Don­nell. Organ­i­sa­tions them­selves esti­mate that they waste 30% of their cloud spend. Giv­en that cloud bud­gets are on the rise, organ­i­sa­tions must ensure they are get­ting a return on this increased invest­ment and allo­cat­ing that spend wise­ly to enable prop­er man­age­ment of their grow­ing cloud ecosys­tem. 

---

---

O’Don­nell adds, “the costs of data move­ment between cloud ser­vice providers and on-premis­es can be a lit­tle bit under­es­ti­mat­ed as well. The big cloud ven­dors are hap­py to let you put in as much data as pos­si­ble and won’t charge you for it. But when you want to take it back out, they hit you with even big­ger charges.”

It’s there­fore real­ly impor­tant for organ­i­sa­tions to be mon­i­tor­ing their cloud usage, cloud ser­vices and cloud work­loads. “Because now we’re look­ing at hybrid cloud, as opposed to one sin­gle cloud oper­a­tor, com­pa­nies cer­tain­ly want to look at a sin­gle win­dow where they can get all those ser­vices and work­loads togeth­er in a sin­gle place; inde­pen­dent­ly from the cloud ser­vice provider’s own mon­i­tor­ing ser­vices,” says O’Don­nell.

A lot of cloud pit­falls also lie in an organ­i­sa­tion focus­ing a short-term effort on cloud migra­tion, and not putting enough thought into oper­a­tions. O’Don­nell notes that this is a bit of a chick­en and egg sit­u­a­tion. “When you’re a chick­en, you’re think­ing about going to the cloud, and then the egg is ‘oh, I’m in the cloud, what next?’ Well, you’re gonna need to get more sys­tems and ser­vices up there.”

Most organ­i­sa­tions get the ini­tial set­up sort­ed and then realise they don’t have much of a strat­e­gy for what to do once they get there and the con­tin­u­al work that needs to be done to ensure suc­cess. And when there’s no clear strat­e­gy, it’s very dif­fi­cult to under­stand what is work­ing and where there is a gen­uine ROI vs over­spend­ing and cost inef­fi­cien­cies. “This is where an invest­ment in an over­all enter­prise archi­tec­ture will pay for itself; dis­cov­er­ing and iden­ti­fy­ing ser­vices, pri­ori­tis­ing based on busi­ness need and plan­ning for the future,” says O’Don­nell.

---

---

This lack of vis­i­bil­i­ty can also cause con­cerns over data secu­ri­ty and prop­er use. This is trou­bling for organ­i­sa­tions, who are increas­ing­ly mov­ing large swathes of crit­i­cal data into cloud ser­vices. Accord­ing to Flex­era, only 16% of organ­i­sa­tions now keep all of their con­sumer data on-prem, and only 23% of organ­i­sa­tions do the same with their finan­cial data. Yet the same report found organ­i­sa­tions cit­ed secu­ri­ty as a top cloud chal­lenge (81% of respon­dents), with gov­er­nance and com­pli­ance also scor­ing high in the con­cerns. 

Effec­tive cloud oper­a­tions must involve an ongo­ing, robust data strat­e­gy that includes data oper­a­tions, data pro­tec­tion and data gov­er­nance. Quest calls this strat­e­gy ‘data empow­er­ment’, a com­bi­na­tion of tech­nol­o­gy and process­es to ensure data in the cloud is migrat­ed and man­aged prop­er­ly, retains its qual­i­ty and is avail­able to every­one and every appli­ca­tion that needs it with­in the organ­i­sa­tion. 

---

---

It’s vital that organ­i­sa­tions have this strat­e­gy in place from the ini­tial cloud migra­tion to ongo­ing main­te­nance and expan­sion. Key con­sid­er­a­tions include: ensur­ing there is a plan for data recov­ery dur­ing migra­tion and once in the cloud, iden­ti­fy­ing and pro­tect­ing per­son­al data pri­or to migra­tion and ensur­ing data trans­for­ma­tion takes into account data type map­pings and oth­er plat­form dif­fer­ences to main­tain data qual­i­ty. Once organ­i­sa­tions are cer­tain of the qual­i­ty and safe­ty of data, it’s para­mount that they have a plan for its effec­tive use. 

O’Don­nell notes that every­thing starts with data, but data in and of itself doesn’t enable an organ­i­sa­tion to suc­ceed. “Data is at the core of every­thing, but its empow­er­ment comes from the infor­ma­tion and the abil­i­ty for con­sumers to con­nect with that infor­ma­tion. And a lot of the work that we do around gov­er­nance is the core of data enable­ment, empow­er­ment and enter­prise intel­li­gence. And at the core of data gov­er­nance is your meta­da­ta at the core of that is your data itself.”

So, if data is at the cen­tre of every­thing, how can busi­ness­es get more out of it? Organ­i­sa­tions must have a rig­or­ous frame­work in place to under­stand their data process­es – what data they are col­lect­ing, where it’s going, how it’s cleaned, how it’s stored and processed, and who has access to it. 

Organ­i­sa­tions must simul­ta­ne­ous­ly con­sid­er the costs of all of this if they are to see a return on their invest­ment. They will need process­es and tech­nol­o­gy in place to test data­base work­loads and assess and com­pare per­for­mance and cost in prepa­ra­tion for migra­tion, and then mon­i­tor infra­struc­ture per­for­mance, resource con­sump­tion and cost once in the cloud.

 

---

---

A cloud strat­e­gy can’t be cre­at­ed and imple­ment­ed in iso­la­tion. It impacts the entire organ­i­sa­tion and so requires input from every stake­hold­er group. Infra­struc­ture may be the obvi­ous focal point, but there is also a gold­en oppor­tu­ni­ty for data engi­neers to future­proof an organisation’s data archi­tec­ture.

O’Don­nell notes the chal­lenge here, giv­en the trend of dis­ag­gre­ga­tion in tech teams. While IT teams will con­tin­ue to stay sep­a­rate, he explains that siloed work­ing must be addressed. “We’re prob­a­bly going to still keep dis­ag­gre­gat­ed func­tions, but work­ing very close­ly togeth­er, mov­ing for­ward. And that’s what’s key to com­pa­nies. They real­ly need to find a rhythm of pro­mot­ing col­lab­o­ra­tion and strik­ing that effi­cien­cy piece. That’s the core of DevOps, site reli­a­bil­i­ty engi­neer­ing, BizDe­vOps – the need to break down silos for effi­cien­cy, pro­duc­tiv­i­ty, qual­i­ty and sta­bil­i­ty.”

With all of this in place, cloud has the poten­tial to bring organ­i­sa­tions firm­ly into the 21st cen­tu­ry. They can ful­ly mod­ernise their oper­a­tions, enable the democ­ra­ti­sa­tion of data and ensure bet­ter busi­ness out­comes to stay one step ahead of the curve in a com­pet­i­tive mar­ket. To get the most from invest­ment in the cloud, organ­i­sa­tions may want to enlist the help of a trust­ed provider. Quest pro­vides solu­tions for today’s press­ing data and infor­ma­tion chal­lenges, includ­ing demands for data democ­ra­ti­sa­tion, data pri­va­cy and cyber­se­cu­ri­ty. Quest can help organ­i­sa­tions build a sus­tain­able foun­da­tion that enables agili­ty, adapt­abil­i­ty and resilience, and max­imis­es the busi­ness impact of their data.

Dis­cov­er how Quest can help you pre­pare, man­age and main­tain your hybrid cloud oper­a­tions

---