DataOps vs DevOps: Uncovering Common Mistakes | DataOps Day

Good morning, good evening, good afternoon to everyone. So I want to express my gratitude to DevOp day 2003 for extending this opportunity to share my thought with all of you. So whether you are season IT professional, student, eager to break into the world of technology or someone curious how is, how is work?

So I'm thrilled you join us today. So today I will love to talk about DataOps versus DevOps and masking the common mistake. So before, before I go to my slides, so I will love to introduce myself.

So I have both software development and IT operation background. So now, as a therefore of for certain and invested, I am senior manager, uh, leading and obvious oversee, uh, therefore adoption and transformation for my client. I'm also a trainer or coach for those want to, uh, embarking, uh, agile plus, therefore transformation for project management.

So during my free time, I also mentoring for people around the world that want to explore in therefore, so this is our agenda for today. So gonna be, uh, introduction, I will, I must all the mistake that probably the organization did or currently doing for the deaf, deaf, uh, for the data ops, uh, journeys. So, and lastly, I will recap our session.

So let's start look at the rise of data ops. So the emerge of data ops. So with the rise of big data and need more agile data management processes, and that's why the data ops was born.

Data ops focus on improving data by streamline data extraction, transformation and loading. And we also call it E T L processes, fostering better collaboration between data scientists, data engineer, and other stakeholder. So I want to discuss about the important in the two days world for data ops.

So agile data management. So the age of big data, being able to rapidly process and direct insight from the data is crucial. So DataOps enable agility so we can improve data quality by streamline data pipeline and collaborative process with in a clean and more available data.

So we also need a faster insight. So reduce the time to take to go on from raw data share number insight, which is in available into this fast space business environment. Okay, let's see what the real world use cases that already implement data up or data pipeline.

So we see, uh, Amazon, they employ data DevOp for software research ensuring, uh, platform remain agile for with DataOps. So they enhance their recommendation in engine that giving personalized suggestion a million of user in real time for, for Google. They use therefore principle to continuous deployment, uh, of service for mine.

Data ops help them to s a algorithm and ensuring users find the most, uh, relevant contents Switzerland. So lastly, uh, look, uh, the, the Netflix. So they leverage DevOp for releasing frequent updates and ensuring uninterrupted stream.

So for the dev DataOps, uh, site, they, they use for help them to analyze a pattern to suggest show and movies that tailor to each, uh, viewer. So this is an example for agile data ops team. So the mission of data ops, it's creating agile, scalable, considerable data flow for data ops team.

And this is a data ops processes with C I C D pipeline. So DevOps encourage, uh, continuous, uh, continuous internet integration, continuous delivery or C I C D process, which is used. All the data activities, it's adds automation to full process of data analytic system and to each part so that change can be made.

And data quality is ensure at each step or stages. So having said that, so many business businesses will like to change their team into agile data ops while others have struggled, uh, to make transaction. So in this session, I would discuss most of the typical error that company make when they're making the switch.

So first common mistake culture more than just a tool. So definition, culture, culture in DataOps and therefore is pretend the quality, value, behavior and beliefs of the team. So influence how they co collaborate, uh, communicate and approach a challenge.

So relevant to this, uh, tool technology can facilitate the processes, but team culture determine adaptability, innovation and our overall effectiveness of operation. So the pitfall on the culture. So, uh, usually, uh, company a tool centric focus.

So over reliant tool can neglect, uh, the important of human interaction, collaboration and team dynamic. So resistant to change a sta culture that hinder education and new methodologists and leading in efficiency and misaligned with their vol evolving into standard. So lastly, the before on the culture, well siloed department.

I think most of the company having this, uh, without collabor collaborative culture department can operate isolation and leading fragment and operation commission breakdowns. So this is the easier, uh, approach that, uh, or any organization can, can apply. So first, monitoring open communication, create environment where the team member feel free to share second, continuous, uh, learning and training.

So encourage ongoing education and skill up your team, uh, can, uh, help to, to fostering the culture. So third one, feedback loop. So regularly, uh, get feedback about the team culture, looking at the area improvement and way to hand on collaboration.

Lastly, uh, leadership role. So leader should add role model and promoting and embody embodying the desire team culture. Okay, now, uh, let's see the second mistake.

So earlier the of new technology usually, uh, organization adopt new based on this two first, uh, quick fix or many organization see new tool as an instant solution to ongoing problem. Second, market height, we detect industry and media promoting the next big thing that's a pleasure to adopt, fully understandable, uh, tool relevance. So this can come up this pitfall.

So meets alignment, the newest tool might not align with the organization actual needs or existing in infrastructure. So second pitfall for the tool complexity. So the introduction of new tool can be unexpected complexity and required additional training and resources last whenever you have a, uh, new tool.

So there will be, uh, resource then like time, money, effort spent on the tool might not deliver, anticipate the r o i or return or investment. So the easier approach that you can apply for the tool, uh, you always, uh, assess whenever, uh, there's a gap or there's a, uh, uh, require for the new tool. So you always do a pilot testing, for example, you're creating a minimum buyer project and then you can try the new tool that you would like to implement.

So lastly, integration planning. So consider how the new thing will fix into the existing, uh, tech stack and workflow. Okay, now I would like to talk about the third mistake, not considering scalability.

So data scalability refer to capacity of system data and process handle growing amount of data and its potential expand to the accom, accommodate that growth. So is it critical aspect system design, ensuring data as we grow the system can maintain its performance, time and efficiency. And let's see, see, uh, what the fit for this.

So short-term vision, so usually organization planning only for immediate needs without anticipate the future demand. So, uh, they also rigid in infrastructure, so updating system or tool that doesn't allow easy scan and mo modification. Lastly, uh, resource constraint.

So organization using, not planning for potential increment or in the resources or the manpower. Lastly, they also not prepared for the finance investment if, let's say they would like to scale, uh, scale to next, uh, size or scale from uh, one cloud to another cloud, okay? This is the easier, uh, approach they can apply.

So first feature proofing. So company always consider what if the user base, uh, user base double or triple when making decision, meaning they can slightly, uh, scale to, uh, one plus, uh, size rather than usually they have. So second, they can apply model of design so they can implement system in the components so they can scale or upgrade in the developer as needed.

Lastly, cloud is a, is a good service whenever they, they use cloud service, which is can offer rapid scalability and offer, uh, with the pair as use grow model. So this is, uh, quite benefit to the organization. Okay, next mistake, uh, that organization always do is the period of extension definition.

So extension in DataOps or DevOp refer to prolong based, where the team stick to the method tool practice defeat, the involve being nature of data management and software development. So the 11 of this data software landscape among the fastest involving field. So if you lay behind it can be bad for the business.

Okay, the pitfall for the stagnant, so usually organization use outdated tool. So that leads slower processing and security is they, they're also using the C I C D, uh, the old or the traditional. So result in longer deployment cycle and miss automation activities.

So they also avoid, uh, microservices, these cost challenges in the scale and managing application. They also have a poor data management. So delay in data insight, a potential informed decision.

Lastly, neglecting security update. So because of the increase of the risk of breaches and non-compliance. So this also, uh, slowing down the data processes.

So the e d approach is you need to have a continuous improvement. So you need to check regularly and re refine your process and tool and practice. So you need to invest in training.

So equip your team with the knowledge of the latest trend and that in DataOps of DevOp, so always doing like p o C to before full scale transaction. The for test for the new tool are method in small project or e, they're affected before you apply it to your production. Okay, last mistake that most organization do whenever they embarking the op uh, journey is, uh, flying with blind with the feedback.

So, uh, organization proceed with without educate information, insight, or respond that inform decision making. So relevant to this feedback is a backbone. It receive development, continuous improvement and data-driven operation.

So the pitfall for this lack of monitoring tool. So, uh, some of the organization not utilizing automated monitoring solution for real time insight. So they also ignoring user feedback, not considering feedback from user or holder can be, uh, can be bad for them.

Then lastly, over relevant on assumption. So usually, uh, they're based on decision on belief rather than the data. This is the e l uh, approaches.

So you need to monitoring and log, have a login tool. So implement tool that offer realtime insight in, into system, uh, performance and user iteration. So second, regularly review session.

You need to have a predict meeting with the stakeholder to gather feedbacks and continuous improvement. Lastly, you need to create a feedback channel. So it make it easy for the user and the team member to give feedback and, and giving, uh, something like giving a pull to, to the, uh, to, to your stakeholder end user.

And then you can, uh, get, uh, fast feedback. Then you can fast improve. We all common being said.

So I like to recap my session. Uh, so data ops versus default, both collaboration, automation and improvement data of focus on data management, DevOps center of software development. So key mistake being, uh, mentioned in the session over emphasize on tool, forget the culture, chasing the trend without sustainability assessment, ignoring scalability and future growth.

The fourth, uh, I mentioned earlier, uh, session is technician. So the, uh, most of designation not, uh, involved with the technology. Lastly, lastly, uh, most of the company are flying blind without any feedback.

So the covid, uh, message that I would like to, to convey success is about people, culture, adaptability, and feedback. That's all my sharing. So thank you for the stay.

If your inquiry, please reach out me via my email or my linking profile. I hope we can cross back again and stay human and continue learning. Thank you.