Category: Uncategorized

5 things I learned architecting AI systems that run tens of billions of inferences per day (That nobody shows in POC demos)

1) “It works in the demo” is the most dangerous sentence in enterprise AI
Most AI initiatives fail in production—not because the model is wrong, but because the architecture can’t survive reality: latency spikes, token limits, context window overflows, partial outages, and cascading failures.

Production isn’t a bigger demo. It’s an entirely different discipline.

2) Real enterprise value starts with multi-agent orchestration
Single-agent demos look impressive. Enterprise problems require 10–15 specialized agents coordinating across domains—with context handoffs, retries, isolation, and graceful degradation.

Designing agent-to-agent communication is 10× harder than building the AI itself.

3) Hallucination mitigation is an architecture problem, not a prompt problem
We implemented automated quality gates before responses reached users:
source verification, confidence scoring, and human-in-the-loop triggers.

This isn’t prompt tuning. It’s production engineering. Result: 97%+ accuracy across tens of millions of daily inference requests.

4) Cost optimization happens at the architecture layer At scale, token costs get brutal—fast. We reduced inference costs by ~40% using intelligent context pruning, hierarchical summarization, and caching strategies—not by switching to cheaper models.

The right architecture makes expensive AI sustainable.

5) The real gap isn’t technology—it’s production experience many AI architects can explain transformers and RAG pipelines.

Very few can explain what happens when:
• an agent crashes at 3 AM
• context windows overflow across a 15-agent workflow
• 10,000 users are waiting for a response

That’s not criticism. It’s the current market reality. Enterprise AI is maturing quickly. And a credibility gap is forming.

The question is no longer: “Can you build an AI demo?”

It’s: “Can you keep it running at 99.95% availability when the stakes are real?”

In today’s rapidly evolving digital landscape, the adoption of generative AI technology is no longer a matter of “if,” but “when.” As a Solution Architect with a passion for staying ahead of technological trends, I’ve been closely observing how generative AI can transform enterprises across industries. From automating content creation to revolutionizing customer service and enhancing product design, the possibilities are vast. But why does generative AI matter so much to enterprises today?

In an era where businesses are under constant pressure to innovate and stay ahead of the competition, generative AI offers a unique opportunity to transform operations, create new revenue streams, and deliver personalized customer experiences at scale. By leveraging the power of AI to generate content, code, designs, and more, enterprises can unlock new efficiencies, reduce costs, and drive growth in ways that were previously unimaginable.

However, adopting generative AI technology isn’t just about plugging in the latest tool and expecting instant results. It requires a thoughtful, strategic approach to ensure successful integration and maximize the benefits. Here’s how I believe enterprises can effectively navigate the journey of adopting generative AI technology.

Start with Assessment and Planning

First and foremost, understanding your business objectives is crucial. It’s not just about jumping on the AI bandwagon because everyone else is doing it. The key is to align AI initiatives with your organization’s strategic goals. Whether you’re looking to enhance operational efficiency, improve customer experiences, or create new revenue streams, identifying the right use cases for generative AI is the foundation of success.

In my experience, starting with pilot projects that target low-risk, high-impact areas can provide valuable insights and demonstrate the potential of AI within your organization. These initial steps help build confidence and create a roadmap for broader AI integration.

Building the Foundation: Data, Infrastructure, and Talent

Generative AI thrives on data, and the quality of your data can make or break your AI initiatives. Ensuring access to high-quality, diverse data and implementing robust data governance frameworks are essential steps. Additionally, decisions around AI infrastructure—whether to go with cloud, on-premises, or hybrid solutions—should be made with scalability and flexibility in mind.

Equally important is building a skilled team. Recruiting AI specialists and data scientists, while also upskilling existing employees, fosters a culture of continuous learning and innovation. In the fast-paced world of AI, staying ahead requires a commitment to ongoing education and development.

Implementation: From Pilot to Integration

Once you’ve laid the groundwork, it’s time to move from pilot projects to full-scale implementation. Integrating generative AI into your existing business processes is where the real transformation happens. Seamless integration with current workflows, coupled with the automation of repetitive tasks, can unlock new efficiencies and streamline operations.

Customizing AI models to fit specific business needs and continuously refining them based on performance metrics ensures that your AI solutions remain relevant and effective. It’s not a one-time effort but a journey of ongoing optimization.

Governance, Ethics, and Change Management

As with any powerful technology, generative AI comes with its own set of ethical considerations and governance challenges. Establishing an AI ethics committee and developing guidelines for responsible AI use are non-negotiables in today’s regulatory environment. Regular monitoring and audits help maintain compliance and mitigate risks.

Change management plays a critical role in AI adoption. Clear communication about the benefits of AI, coupled with support for employees during the transition, can alleviate concerns and drive smoother implementation. It’s about fostering an AI-friendly culture that embraces innovation while addressing potential fears and misconceptions.

Continuous Improvement and Long-term Vision

The journey doesn’t end with the successful implementation of AI. Regularly monitoring AI performance against key metrics and gathering feedback from users are crucial for continuous improvement. Staying updated on the latest AI developments and being open to new opportunities allows your organization to remain competitive and innovative.

Looking ahead, it’s essential to have a long-term AI roadmap that aligns with your overall business strategy. Anticipating future AI trends, investing in research and development, and maintaining flexibility in your AI strategy will ensure that your organization is well-prepared for the future.

Collaboration and Partnerships: The Power of Together

Finally, collaboration is key. Internally, fostering cross-functional teams and encouraging knowledge sharing can accelerate AI adoption. Externally, partnering with AI vendors, industry consortiums, and research institutions can provide access to cutting-edge technologies and best practices. It’s through these collaborations that enterprises can truly unlock the full potential of generative AI.

Adopting generative AI technology is a strategic journey that requires careful planning, robust infrastructure, ethical considerations, and continuous learning. By following this approach, enterprises can not only harness the power of AI to drive growth and efficiency but also position themselves as leaders in the digital age.

I’m excited to see how Generative AI will continue to evolve and transform the business landscape. If your organization is considering AI adoption, now is the time to start laying the groundwork. The future is bright, and the opportunities are endless. Let’s embrace the power of AI together.

The Oracle Cloud Infrastructure (OCI) 2020 Architect Associate exam is one of the demanding certifications for 2020/21 as the OCI is growing rapidly and even Gartner reported twice as OCI is the visionary in public cloud platforms. OCI is unique in its reliable infrastructure, cost and support. As like other cloud service providers, OCI is having many cool features like data guard, fault domains, compartments and their biggest asset is “Autonomous Database”

Here are the topics I am covering in this post

1. Certification details (Cost, Registration process and Exam topics)
2. Examination preparation materials and my reviews
3. My experience through exam preparation and exam day
4. Syllabus in detail to prepare for the exam

1. Certification details:- 
   1. The exam duration is 85 minutes for a total 60 questions, where you need to score 65% in order to pass the exam, i.e 39 correct answers will get you certification. The examination costs you USD 150. See below link for more details.

https://education.oracle.com/oracle-cloud-infrastructure-2020-architect-associate/pexam_1Z0-1072-20

• Examination consists of multiple choice questions only, where you will have 4 or more answers to select from for every question. Some questions you may need to select 1 or more suitable answers. There will be a question or two which shows a graphic and asks the question (I will explain more in next sections)

• In order to register for the exam, you need to have an Oracle account and login through Certview to pay and schedule the exam. Below the link for registration

https://catalog-education.oracle.com/pls/apex/f?p=1010:26:2076046940597.

• You can attend the exam either at home or going to a PearsonVue test center and the exam price is the same in either of the cases. You will get the exam results immediately as pass or fail. Once you pass, you need to wait for 24 to 48 hours to receive your certification to your Oracle account. You will get the digital badge through Acclaim. 

• Exam topics cover 5 major areas. Exam verifies your knowledge in architecting various solutions by leveraging OCI services in a best possible way (Low cost, resilient, reliable, high performance and secured)

1. Identity
2. Networking
3. Compute
4. Storage
5. Databases

2. Examination preparation materials and my reviews:- Below the learning subscription from Oracle University. This subscription consists of workshop, examination preparation, practice test and exam voucher but it will cost more than 3000 USD for the entire package. I would recommend go through the freely available option in this subscription those are actual workshop and practice test only

https://learn.oracle.com/ols/learning-path/become-oci-architect-associate/35644/75658

This course will help you understand the various OCI services like compute, network, storage, containers, identity, security etc but will not provide the actual guidance to clear the examination like best practices, architectural patterns, design considerations, lessons learned, various use cases and time/situation based decisions which are required to pass the examination. I will cover how you can gain such knowledge in the upcoming sections

b.  The Official OCI documentation: Below the link for official OCI documentation that covers entire OCI services and features in depth.  

https://docs.oracle.com/en-us/iaas/Content/home.htm

I strongly recommend reading the Compute, IAM, Networking, Security, Storage, Databases sections in order to better prepare for the exam.

c.  The book – “Architect Associate All-in-One Exam Guide (Exam 1Z0-1072)” published by  ‘McGraw Hill’, authored by Roopesh Ramklass. I also strongly recommend this book as it provides over all details of each OCI service that is required for the associate architect exam. 

Note:- Don’t miss the “Tips” section in this book which is completely examination oriented. These tips are also a game changer to pass the exam.

d. The Youtube Channel:- Below the link for the Youtube official OCI channel

https://www.youtube.com/channel/UC60OcDzeEtn194-UPYNJs8A

There are great videos especially “Networking” and “Core Concepts” sections, which you can refer while preparing for the exam. Check other videos as well as they cover out of the box concepts in adopting cloud/OCI and other industry related information.

e. Explore and make your hands dirty with OCI free tier account. Practicing OCI will make you perfect in preparation to the exam. Below URL is for free tier OCI

https://www.oracle.com/cloud/free/

3. My experience through exam preparation and exam day: I am a 6x AWS certified, CCSK and CCSP with 7 years of cloud computing experience in Cloud Security, Adoption, Migrations, Automation and DevSecOps, so this over all experience helpmed at least 30 to 40 % of OCI architect associate exam preparation. I have undergone all above said 5 different types of learning ways for about 35 days (each day 1+ hour and 2 to 3 hours during weekends). During the normal  business days, it is a little difficult to focus on studies, being full time employed and being a fulltime husband, father 🙂 

I have booked the exam at a local PearsonVue testing center (Knowing my kids may disturb me if I take the exam from home 🙂 ). I am ontime to the test center and finished check-in formalities and directed to my hot seat. 

The very first question I saw,  I thought I was going to fail the exam, as the question was two paragraphs and took 3+ mins to read and understand. In fact I even don’t have the right answer. My Tip here is – Don’t Panic if you don’t know the answer. If I don’t know the answer I try to guess the best possible option with my experience in Cloud Architectures. However there are not many questions that are lengthy but most of the questions are two to 3 lines of questions only. 

There the below types of questions I encountered 

1. Question with multiple choice answers which you have to pick one right answer

2. Question with multiple choice answers which you have to pick more than one right answer

3. Question with a graphic with multiple choice answers which you have to pick one right answer.

                      Note: There are no “True” / “False” questions that I see in my exam

 Time is also an essential part to consider, I have 5 mins left to answer 9 questions in my exam.  So you need to be cautious when you are spending more time on questions you are uncertain. Keep them in review stage and move on to the next question, so you can revisit the review marked questions in the last and think about them.  Finally I have submitted the exam right before it auto closes the session and I see the result as “Pass”. Hurray, I did it. Remember the core secrets of my success are my previous experience, OCI official documentation reading and the All-in-One book (specially the tips in the book).

Now you have everything you need to know, and it’s time for you to plan (based on your previous experience in Cloud/OCI), what to study, how to study and how long to study. Let me know if you have any additional questions. All the best !

Is your migration to the cloud being a daunting process ?. Then its time to make it done in the right way to ensure it supports your business case and delivers the value that is expected since migration infrastructure, data and applications will affect both people and processes that rely upon them.

Planning is key to the potential successful migration to the cloud and you need to create a migration strategy by identifying the potential migration options, understanding the inter dependencies between applications, data and infrastructure.

Effective planning of cloud migration have 6 R best practices that enterprises are successfully practicing. But make sure to discuss these points with application/data owners to prioritize the assets to migrate with different strategies.

Rehost (Lift & Shift, Fork Lift) – simply ‘clone’ your servers and move them to the cloud provider’s Infrastructure. The Cloud Provider will then manages the underlying hardware and hypervisor infrastructure, and you continue to manage the same operating system and installed applications.

Repurchase – Many commercial applications are now available as Software as a Service (SaaS):Example Salesforce. So it may be easy to you move away from managing applications and infrastructure and adopt SaaS deployment model.

Replatform – Moving your application currently relying on legacy infrastructure to the new cloud based infrastructure. You are only changing underlying services, while maintaining the core application code, so the architecture of your applications will not change.

Refactor – Make changes to your application code to leverage cloud native services. Simply called “application modernization”. Example : It may be that you want to move away from server-based applications to leverage cloud provider serverless functionality

Retain – Your on-prem applications may be newly developed with newly invested money and other resources may not satisfy the business case to move to the cloud or the application dependencies with their software licenses may not yet support the public cloud platforms. So you will retain those applications back in on-prem.

Retire – You must classify what applications/infrastructure needs cloud migration and get rid of the aged/no business required assets. Simply retire the apps and infra instead moving them to cloud by spending time and cost.

As I always define the benefits of cloud computing in 3 ways

1. Business Strategy (Shared responsibilities between consumer & CSP, fearless Updates & Upgrades, Global collaboration, Technological transformation & ahead in the competition)

2. Technological Flexibility (Infra Scalability, storage options, deployment choices [IaaS, PaaS, SaaS], features and tools)

3. Operational Efficiency (Global accessibility, Infra/App/Data security, agility that speeds to market, CapX to OpeX, OnDemand that pay as you go)

We know the above cloud computing benefits for many years by now, but to my experience being a Cloud Solution Consultant, handled many customers across the globe and below the consolidated list of questions from the C level customer executives that I met. They are all about cloud adoptions and migrations

1. How public cloud is better than On-premise?

2. How complex the migration of our corporate data center to public cloud?

3. Are the current applications deployable to cloud? and how can we assess them?

4. What is the cost for the infra/app migration?

5. What will be the TCO and ROI benefits if we move to the cloud?

6. Is public cloud secured than on-premise?

7. How our competitors using public cloud?

That is why I was engaged? Yes, what I could say to these precious clients is

As we migrate to the cloud with some essential steps which can evaluate the situations and provide a detail which can address the above questionnaire and a road map to the future public cloud establishments.

I always say Initiate, Assess, Approve, Build & Migrate

Initiate: Enterprises have to initiate the plan of moving to Public Cloud as per their desired vision by bringing all the technical/Business stakeholders into one platform and kick start the planned steps (Something like — engage a strategic partner, build a CoE team, acquire the knowledge etc…)

Assess: Do an assessment of current state of infrastructure, applications, data, security & networking of on-premise over public cloud platforms

Approve: Getting the plans approved is typical for any organization. Plan must adhere to the organizational policies, regulations and compliance requirements. Business needs to approve the cloud adoption plan

Build: Build a Demo/POC environment in Public Cloud which can support the infra/app/data hosting

Migrate: Move the simple workloads first to the cloud and test the benefits.

Later leverage the tools & services of particular CSP to build a secure, reliable, efficient cloud platform which can support the 6 Rs (Re-hosting, Re-platforming, Repurchasing, Refactoring, Retire & Retain)

So now I advised my customer the benefits of cloud computing and the strategic steps to take to migrate to the cloud. But what is the approach or method to follow?

That is where the “Minimum Viable Cloud” approach comes to ……

There are many use cases for an organization to move their workloads to Public cloud platform. Under any circumstances or use cases, building a secured public cloud platform which can satisfy the requirements to launch at least one application and engages all the key stakeholders of that organization, where they can explore, experiment that platform to their need….is called the minimum viable cloud environment and the approach is nothing but MVC approach.

1. This cloud platform should demonstrate the viabilities of cloud services

2. This cloud platform should be secured in terms aligning with Cloud security alliance guideline, and ensures the infrastructure/application/Data/Network security

3. This cloud platform may have Agile mindset, Automation, DevSecOps, Ci/CD of Infra/App pipelines, Virtual/Physical connectivity with on-premise data center & extended corporate user access.

During this MVC approach, customer should be ready with the details to satisfy above stated MVC characteristics.

Below few use cases to MVC approach

1. Deploying a web page with high availability and geographical accessibility

2. Re-hosting an enterprise application with logging & monitoring in place

3. Migrating a Sybase DB to Cloud MSSQL

4. Re-engineering a database to host in public cloud with high security and low cost

5. Building a multi environment platform with CI/CD for faster application releases.

MVC approach is a secure and comprehensive way of cloud adoption!!!

The “Certificate of Cloud Security Knowledge (CCSK)” is the first professional certification in Cloud Security Industry released in 2011 and gained the momentum very soon. If you see the top vendor neutral certifications for Cloud Security, then CCSK & CCSP (Certified Cloud Security Professional) stands ahead of other certifications.

In 2017 the CCSK version has changed from version 3 to 4 and the current examination will test the knowledge over version 4 only. The CCSK is managed by “Cloud Security Alliance” and CCSP is being managed by (ISC). CCSK helps individual cloud security engineers and architects, employers, cloud service providers and consulting firms with to develop the best security programs related to globally accepted standards and further build and maintain the secure cloud business.

How to prepare for the CCSK exam:

There could be multiple ways to prepare for the exam, but below my thoughts may ease the preparation and clear the certification at the first attempt.

Step 1: – Read and understand the Security guidance provided by the Cloud Security Alliance. This can be freely downloaded from CSA website else you can find one from the preparation kit that I have shared in this post. This guidance book consists of 152 pages with 14 security domains defined. Below the information that how I received the examination questions across these domains but may vary slightly case by case.

Domain 1: Cloud Computing Concepts and Architectures (6 questions)

Domain 2: Governance & Enterprise Risk Management (2 questions)

Domain 3: Logical Issues, Contracts & Electronic Discovery (3 questions)

Domain 4: Compliance & Audit Management (3 questions)

Domain 5: Information Governance (2 questions)

Domain 6: Management Plane & Business Continuity (4 questions)

Domain 7: Infrastructure Security (6 questions)

Domain 8: Virtualization & Containers (5 questions)

Domain 9: Incident Response (4 questions)

Domain 10: Application Security (6 questions)

Domain 11: Data Security & Encryption (6 questions)

Domain 12: Identity, Entitlement & Access Management (3 questions)

Domain 13: Security as a Service (SecaaS) (2 questions)

Domain 14: Related Technologies (1 question)

I strongly recommend reading this document at least 2 times thoroughly. There are at least 80% of the questions are direct statements from this book. Below is an example

Question: – Immutable workloads make it faster to roll out updated versions because applications must be designed to handle individual nodes going down

1. True
2. False

The right answer is “True”.

You can see the same statement written in the Security Guidance Book page 86, below the screenshot of the same.

Step 2: – Read and understand the “ENISA” (European Network & Information Security Agency) document of benefits, risks and recommendations for information security (Mainly the ‘Top Security Risks’ Section)

I received 3 questions from this section in the CCSK exam. I have shared this document in the preparation kit as well in this post.

Step 3: – Walk through the Cloud Control Matrix (CCM v 3.0.1)

The Cloud Controls Matrix is a baseline set of security controls created by the Cloud Security Alliance to help enterprises assess the risk associated with a cloud service provider

I have received 4 questions from this section and shared the CCM document in this post as part of the preparation kit.

Preparation kit : Below the link for the preparation kit

https://github.com/spadigala/CCSKv4

Examination Pattern, Tips and Cautions

Exam Pattern: – CCSK exam can be registered through the below link by signing up.

https://ccsk.cloudsecurityalliance.org/en/login

• Once you sing up, you need to buy the exam with credit card or paypal which costs USD 395. You will receive 2 attempts to acquire the certification and if you pass the exam on your first attempt, then the second attempt will be wiped out.
• CCSK exam is very flexible that you can attend this from your home.
• CCSK exam consists of 60 questions (Multiple choice and True/False) with 90 minutes of time to complete. You can ‘Mark for review’ the question you like to revisit, but if you mark the question for review and unable to review them in the given 90 minutes time, then these questions will not have calculated to the results

Exam Tips: –

I suggest attending the exam with a laptop or desktop having 2 monitors. One monitor you can attempt the exam and another monitor you can open the guidance documents like ENISA, CCM etc. where you can search for the words listed in the question at these guidance documents for references.

Cautions: –

Time is very crucial in this exam as many thinks that this exam can be attended at home by keeping the required material open to read and search which a waste of time in many cases as you may search and answer the 50% questions but will fail to attempt the remaining 50% as the time lapses.

Available trainings from the market

• CSA (The Cloud Security Alliance) provides the information regarding Classroom/Virtual trainings across the globe. You can search the same in below link. But keep in mind that these trainings are very expensive as I could see the minimum training cost is USD 1945 for a 3-day training program.

https://intrinsecsecurity.com/training/courses/ccsk/

• There is an OnDemand online training (Recorded video) provided by https://intrinsecsecurity.com/ but still it will cost you USD 995. But one good thing is that you will receive an examination voucher to redeem where you will save USD 395
• You can buy and practice the questions from Whizlabs (https://www.whizlabs.com). Note that these questions are to self-assessment purpose only and not the examination dumps. Do not expect more than 5% of questions in real exam from these. But this practice questions can help you prepare for the exam by building you the confidence on the required subject

In my view, if you can undergo the above stated 3 Step examination preparation for about a month of time, then you do not need any of these trainings as these trainings help you only visualize the content than reading the guidance document.

Once you finish the exam, then you will receive the results immediately either pass or fail and detailed report of the results containing each domain and questions received and answered correctly. You can download the pdf version of the certificate immediately after you pass the exam.

All the best to your exam and let me know how my plan helped you for the exam.

In late 2016, when I was stuck at building a CI/CD pipeline with security controls towards a PaaS solution to one of the largest UK bank, I used to think that there should be a philosophical approach (in every organization) to make everyone feel responsibility of security in what they are building and innovating. In late 2017, while I was engaged with a large insurance client, I was requested to identify and build security capabilities as an interim solution to an App/Infra CI/CD Pipeline, again i got stuck with lack of blue prints and playbooks for each service in pipeline to integrate security. For example a Jenkins playbook which contains a detailed operating reference with security in place (For CI solution).

I seriously though its a philosophical/cultural shift to an organization to enhance their DevOps practice with security. And in early 2018, I learned the term – DevSecOps, which fulfilled my English to my previous experiences and thoughts. Yes the name defines it. Practice your DevOps culture with an attitude of Security as Service (SecaaS) and Security as a Code (SecaaC). Both SecaaS/SecaaC defines your organizations operational and engineering efficiencies. Thats ‘DevSecOps’

DevOps is not only about developing and operating business and teams. As I always say adopt DevOps culture and agile mindset, but if you want to see a better outcome of those two attitudes, then security must also play an integrated role in the full life cycle of your infrastructure and applications. Traditionally Developers, Operations team and Security team are different. Now its like a one man army – The “DevSecOps Engineer”

As everyone know that DevOps/Agile will get your code developed and released faster and frequent, but with lack of security standards (Or legacy standards) can ruin the business goals and fails the DevOps approach.

I suggest the developers, ops guys and sec engineers to maintain short and frequent development cycles, and integrate security measures (try to reduce/minimize Operational disturbances by doing so),
speed up innovative technologies like containers and microservices, and all the program manage/scale the DevSecOps approach between commonly isolated teams—this is a tall order for any organization.

Next time, I will comeback here to post more glimpses of pipeline (Infra/App) components with security integrated and will publish some playbooks too.