Python has emerged as the top programming language for developers building generative AI applications, according to new research.
“Python is the language of choice for AI programming,” according to a report by cloud data company Snowflake, which analyzed usage data from 9,000 of its users.
It saw a 571% growth in Python usage on the Snowpark platform, significantly more than any other language year over year. The use of other languages also increased – such as Scala (up 387%) and Java (up 131%) – but not as quickly.
“Python skills will be increasingly essential for development teams as they venture into advanced artificial intelligence,” the report said.
That’s because Python has a lot going for it, the report says. The programming language is easy to learn and read, “allowing developers to focus on solving AI problems rather than parsing abstract syntax.”
It has a large ecosystem of libraries and frameworks to simplify otherwise daunting AI tasks, “from implementing neural networks to natural language processing.” And there is also a large active community of collaborators who help in learning and solving problems.
“In general, Python allows developers to focus on the problem rather than the language. They can work quickly, speeding up prototyping and experimentation — and thus overall learning as development teams get into cutting-edge AI projects early,” the report said.
Snowflake’s research follows analysis earlier this month that showed enthusiasm for Python continues to grow. The Tiobe Programming Language Rankings noted that the gap between Python and the rest of the pack has never been wider.
Python supports unstructured data enhancements
The Snowflake report also found that enterprises are tapping into their unstructured data. Most data — perhaps as much as 90% — is unstructured, in the form of videos, images and documents, but the company said processing of unstructured data grew by 123%.
“That’s good news for many uses, not the least of which is advanced artificial intelligence,” the report said. “Proprietary data will favor large language models, so unlocking that underutilized 90% has great value.”
In particular, these types of data are processed with Python, Java and Scala.
“Given that Python in particular is the language of choice for many developers, data engineers and data scientists, its rapidly growing adoption suggests that these unstructured data workflows are not just for building data pipelines, but also include AI applications and ML models,” she said. is a snowflake.
Snowflake captures the “LLM explosion”
Certainly, building AI-powered generative applications based on large-scale language models (LLM) is now a priority for many developers.
“There’s an LLM explosion going on right now—probably in your office,” Pahuljica said.
The company said it saw 20,076 developers working on 33,143 LLM-powered apps in its Streamlit developer community last year. Nearly two-thirds of developers said they were working on work projects.
While generative AI has yet to become a mainstream technology, Snowflake said “we’re definitely seeing a lot of effort to get there as soon as possible,” highlighting the intense enterprise interest in using AI tools and applications.
The type of apps developers are building is also evolving – Snowflake said between May 2023 and January 2024 at Streamlit, chatbots grew from 18% of LLM apps to 46%.
This most likely does not represent a shift in market appetite for LLM applications, but it does show how developers are increasing their skills and can build more complex chatbot applications.
The developers said their main concern when building generative AI applications was whether the LLM answer was correct – a reference to the ongoing issue of AI hallucinations – followed by data privacy concerns.
In addition, companies are also taking a more proactive approach to data management. The number of tags applied to an object increased by 72%, while the number of objects with a directly assigned tag increased by nearly 80%, and the number of masking or row access rules applied increased by 98%.
But Snowflake also said that the cumulative number of queries launched against policy-protected objects increased by 142%. This is particularly significant, it said, because it showed that companies are increasing their use of data while ensuring responsible use.
“We’re seeing more and more governance using tags and masking rules, but the amount of work being done with this more carefully managed data is growing rapidly.”
Jennifer Belissent, chief data strategist at Snowflake, said data security has long been in the spotlight, but the rapid acceleration of AI applications has brought the issue to the fore. Addressing issues like privacy and security “brings peace of mind.”
“When data is protected, it can be used safely,” she said.
“Taken separately, each of these trends is a single data point that shows how organizations around the world are dealing with different challenges. Taken together, they tell a larger story about how CIOs, CTOs and CDOs are modernizing their organizations, artificial intelligence experiments and data problem solving — all necessary steps to take advantage of the possibilities provided by advanced artificial intelligence.”