Today, we engage in conversation with Kirill Sergeyev, Lead Backend Developer at AWWCOR. With over 15 years of experience in development, Kirill holds a degree in information security.
Throughout his career, Kirill has delved into various neural network technologies, contributed to the development of products intertwined with BigData, and navigated the currents of handling massive streams in the realm of HighLoad (streams exceeding 10 Gb/s, rps surpassing 150K). Lately, Kirill has been instrumental in crafting the backbone of the company’s primary system, a system that lends its support to pharmaceutical endeavors.
Greetings. It is incumbent upon us here to elucidate the very notion of AI and our conception of AI technologies. In essence, the true artificial intelligence, the one spoken of as destined to conquer the world, is more aptly termed AGI – Artificial General Intelligence.
Greetings. It is incumbent upon us here to elucidate the very notion of AI and our conception of AI technologies. In essence, the true artificial intelligence, the one spoken of as destined to conquer the world, is more aptly termed AGI – Artificial General Intelligence.
What we currently term AI technologies are, for the most part, tools we employ to tackle specific challenges that elude resolution through conventional computational algorithms. Primarily, these are tasks where algorithmic discovery falters due to the intricacies of the connections between input and desired output. Alternatively, they arise when the algorithm is clear, yet constraints upon data or performance come into play.
Over the course of my career, I have employed AI technologies to address a multitude of challenges. In the early days, these encompassed decision tree technologies (ranging from conventional trees to XGBoost, LightBoost). Subsequently, ensembles of decision trees found their utility – predominantly, I harnessed them for the analysis of questionnaires completed by operators during cold calls. This proved to be an immensely valuable feature, significantly expediting the aggregation of cumulative results for the business.
Further, I delved into recursive neural networks for the prediction of information and loads for successive blocks within various blockchains. This information was imperative to allocate suitable.
Additionally, there was a phase during which I employed the renowned convolutional networks to analyze images and extract semantic information – though the scope was contingent upon the exigencies of the business and the product at hand.
Furthermore, there was a period during which I employed renowned convolutional networks to analyze images and extract semantic information.
Presently, we deploy neural networks in diverse manners within our operations: aiding in the swift formulation of research outputs based on input data, assisting in predicting data gaps and trends.
Moreover, we are currently engaged in the integration of specific LLMs aimed at addressing various textual tasks, such as composing research texts, graph descriptions, errors, and other data that help researchers in the pharmaceutical field interact with aggregated data, relying more on ordinary English language than artificial tools.
How do you currently assess, Kirill, the extent to which the application architecture, existing infrastructure, and development tools are prepared for the widespread implementation of neural networks in products? Could it not transpire that what appears exceedingly enticing at the ideation stage metamorphoses into sluggish products with occasional successful outcomes, causing users to revert back to the old, dependable, and swift non-neural network products?
Indeed, the matter you are addressing is indeed pertinent. I have encountered various scenarios both in my workplace and elsewhere, wherein attractive results are showcased during demos, yet when the product is rolled out into production and delivered to the user, they transform into expensive novelties rather than tangible products. Naturally, counterexamples can be furnished – for instance, Visual Studio and CoPilot.AI, both of which numerous programmers employ to expedite their work. I am also aware that designers employ tools to streamline their tasks, and business analysts often employ AI technologies to prepare reports and presentations.
However, as soon as these endeavors enter the mass market, I observe a plethora of questions emerging, the resolution of which currently poses distinct challenges.
The first question pertains to the accuracy of outcomes. When we discuss neural networks, they still provide answers with a certain degree of probability. Researchers endeavor to elevate this metric to a high level, but firstly, it never reaches an absolute 100 percent. What if a particular user happens to be part of that 1 percent where errors occur? The law of large numbers comes into play, and thus we encounter a dilemma with a constant 1-5 percent of clients whose metrics, which may diverge from the actual truth, and adjusting them on-the-fly can pose significant challenges. Yes, fine-tuning neural networks can be utilized, but I have witnessed many instances in practice where fine-tuning markedly diminishes the quality in a production environment. Once again, I am not asserting that these challenges are insurmountable, but they are indeed present.
The second question, and a crucial one, relates to the issue of harm. This is classically illustrated through the example of controlled AI machines. If that fateful percentage of error occurs, the consequences can be dire. Now envision a neural network generating stories for children, and suddenly it errs, hallucinating, and starts indoctrinating them with fascism. This issue has long been under scrutiny, and various major firms like OpenAI, Claude, and others are grappling with it using different constraints.
Nonetheless, the problem persists, and it is imperative to remain attentive to it and possess the means to address it, for the repercussions could be catastrophic for the product.
The third question pertains to performance. If we possess an excellent neural network, but at times, it provides answers in 100 milliseconds and then in 3 seconds, unpredictably, this will dissuade users from engaging with the product.
The fourth question revolves around testing. When you have refined algorithms, have tested them under specific circumstances, and gained an understanding of your product’s quality, it is crucial to recognize that we now live in a world where products fully utilizing AI can yield specific results on test data, yet there is no certainty that they will produce the same outcomes on actual data. Hence, it becomes essential to meticulously prepare for tracking the subsequent evolution of the product in a production environment.
In essence, AI can expedite the development process, especially with fine-tuning, effectively laying to rest the waterfall approach to development where products were painstakingly crafted, tested, and only then launched. What was once managed using various Scrum and Agile techniques, surviving within the cycle of short releases, now requires even more frequent and precise responsiveness to issues in production, each time realizing that quality cannot be guaranteed. AI compels the entire product to exist within a realm of probability while granting the capacity to address a greater array of situations than conventional algorithms.
Yet, all of this presents an immense array of challenges for developers and architects, demanding a thoughtful response to the aforementioned problems. And this is just a fraction of the challenges at hand.
This leads us to the question – are the tools in architecture and development currently ready to ensure a seamless release of products entirely based on AI, while guaranteeing a certain level of quality?
Formally, they are prepared. There are numerous utilities and ample tools available to assist an architect in constructing a system that deploys a neural network and monitors its quality in conjunction with the other resources comprising the product. However, in practice, we encounter several challenges that impede the swift release of many products in the here and now, as well as the instant transition from research to full-fledged product deployment and support.
The architecture for AI-utilizing products differs from the architecture of traditional products, which have been honed through years of practice to a nearly faultless state. Undoubtedly, the experience accumulated from conventional methodologies will prove useful for AI-based products. Nevertheless, in practice, we witness the emergence of new types of architectures and CI/CD processes tailored for product deployment.
First and foremost, this pertains to quality control, as discussed earlier. For AI, it takes on a different dimension, closely tied to the data upon which the system’s functionality must be validated. In doing so, we must acknowledge the fluid nature of data and necessitate an entire team to ensure that the data genuinely reflect the real quality of the product.
Secondly, there are the data themselves – often, preprocessing and post-processing of data play a crucial role in the product. These data flow in real-time, in significant volumes, and often in an aggregated state. This gives rise to substantial engineering challenges in establishing data consumption and production processes, more intricate than those encountered in conventional products.
Thirdly, there is the model deployment process. We must somehow ensure the continuous deployment between versions of released models and the entire product infrastructure that employs them.
Even if we possess an exceptional model, a misalignment between the model and the infrastructure, such as the incorrect protocol, can engender significant issues.
In essence, while the formal readiness of tools and architectures may exist, the practical application of AI in product development reveals a myriad of intricate challenges that necessitate novel solutions and practices. The shift towards AI-driven products requires a holistic rethinking of conventional approaches, from data.
Fourthly, there is the matter of monitoring. We recognize that, often, a product involves the interaction of multiple external APIs, diverse types of neural systems, and data streams that split during the process of prediction or generation. This necessitates user-friendly and comprehensible monitoring tools, not solely for specialists.
Although such tools abound in the market, configuring and deploying these systems in a manner that takes into account the specificities of your product and enables real-time tracking of output data is a labor-intensive task and an architectural challenge at the current juncture.
Fifthly, we confront the challenge of support. The entire flow, from encountering an error to comprehending its true origin, is frequently complicated by the fact that the neural model is a black box to us.
And what if there are 5 or 6 different types interacting within a project? We had a project where random forests interfaced with LSTMs, and the output was refined through regular 5-layer neural networks. Additionally, constant fine-tuning specific to individual users was integrated. When incorrect output data emerged, no one could definitively ascertain what had gone awry.
The intricacies of troubleshooting such systems are exacerbated by the complex interplay of various model types, and the opaqueness inherent in neural networks compounds the challenge. This demands an elevated level of technical expertise, time, and resources to fathom the underlying reasons for malfunctions and to rectify them effectively.
In conclusion, while the architectural potential and available tools might nominally support the integration of AI into products, the practical reality unveils a labyrinth of hurdles that extend beyond the theoretical boundaries. From quality assurance through deployment, monitoring, and support, the AI-driven product lifecycle necessitates a comprehensive reevaluation of existing methodologies and a proactive approach to addressing emerging complexities. Only by meticulously confronting and resolving these multifaceted challenges can the seamless integration of AI become a tangible reality, enabling the creation of products that harness the full potential of artificial intelligence while delivering consistent quality and user experiences.
Sixthly, another challenge arises when it comes to security. Often, in the products I have worked on, the primary advantage of neural models lies in their customization for individual users. Yet, we cannot retain user data or disclose it in any form. However, for error handling, especially specific errors that arise in the process, such as generative text LLMs, we require input data. As a result, we have arrived at a solution where we primarily retain aggregated and de-identified data, or based on these, attempt to generate test sets in the event of errors. These test sets are then used by the development and research teams to troubleshoot problems.
Seventh and lastly, a challenge I continually face is performance. The issue arises when accuracy demands more time for the execution of neural networks and greater memory consumption. Conversely, there are constraints on response time and device memory for end users. We endeavor to apply traditional techniques to accelerate response times. This includes intelligent caching and simpler responses from basic models, which are later substituted frequently, we have limited control over the models themselves.
Thus, we collaborate with data science to streamline models and reduce their execution duration without compromising significant quality. Often, not everyone grasps that response time is just as vital as a prediction or generation accuracy.
In conclusion, we approach the culmination of our interview. So, are the development and deployment tools for AI-driven products ready for widespread application in the present day?
They exist, and there is quite an abundance of them. This is already promising. I won’t delve into specifics, but prominent FAANG companies possess commendable tools, and numerous startup products boast worthwhile resources as well. However, at the moment, architects and system designers must fine-tune these tools for each project, navigating through trial and error to find suitable solutions. The predicament lies in the ever-changing and volatile nature of data. While neural models are already adapted to this dynamic environment – that’s their essence – the development and delivery means for ensuring acceptable quality and performance are still a work in progress.
I believe that in around three years, well-established usage patterns, time-tested development tools, and a glossary of terms will emerge. This lexicon will signify how to seamlessly facilitate AI-based product operation from scratch, ensuring the desired quality, responsiveness, and continual deployments without unnecessary obstacles.
