Open-Source AI Is Increasingly Popular But Not Risk-Free – InformationWeek
TechTarget and Informa Tech’s Digital Business Combine.TechTarget and Informa
Together, we power an unparalleled network of 220+ online properties covering 10,000+ granular topics, serving an audience of 50+ million professionals with original, objective content from trusted sources. We help you gain critical insights and make more informed decisions across your business priorities.
Open-Source AI has a raised profile due to its growth in exposure and popularity, but it is not without its own procedures, challenges, and idiosyncrasies governing its adoption and use.
February 28, 2025
Open-source AI projects are exploding in popularity and are contributing to PwC’s estimated $15.7 trillion impact AI will have on the global economy by 2030. However, some enterprises have hesitated to fully embrace AI.
In 2023, VentureBeat found that while more than 70% of companies were experimenting with AI, only 20% were willing and able to invest more.
Open-source tooling offers enterprises cost-effective, accessible AI use with benefits including customization, transparency and platform independence. But it also carries potentially hefty costs for the unprepared. As enterprises expand their AI experimentation, managing these risks becomes critical.
Many AI tools rely on vast stores of training data to develop models and generate outputs. For example, OpenAI’s GPT-3.5 was reportedly trained on 570 gigabytes of online text data, approximating 300 billion words.
More advanced models require even larger and often less transparent datasets. Some open-source AI tools are released without dataset disclosures or with overwhelming disclosures, limiting useful model evaluations and posing potential risks. For example, a code generation AI tool could be trained on proprietary, licensed datasets without permission, leading to unlicensed output, and potential liability.
What Netflix’s ‘Zero Day’ Got Right (and Wrong) About Cyberattacks
Open-source AI tools using open datasets still face challenges, such as evaluating data quality to ensure a dataset hasn't been corrupted, is regularly maintained, and includes data suited for the tool’s intended purpose.
Regardless of the data’s origins, enterprises should carefully review training data sources and tailor future datasets to the use case, where possible.
Proper data, model, and output licensing presents complicated issues for AI proliferation. The open-source community has been discussing the suitability of traditional open-source software licenses for AI models.
Current licensing ranges from freely open to partial use restrictions, but unclear criteria for qualifying as “open source” can lead to licensing confusion. The licensing question can trickle downstream: If a model produces output from a source with a viral license, you may need to adhere to that license’s requirements.
With models and datasets evolving constantly, evaluate every AI tool’s licensing against your chosen use case. Legal teams should help you understand limitations, restrictions and other requirements, like attribution or a flow-down of terms.
How to Overcome the Quantum Threat
As global AI regulations emerge and discussions swirl around the misuse of open-source models, companies should assess regulatory and privacy concerns for AI tech stacks.
At this stage, be comprehensive in your risk assessments. Ask AI vendors targeted questions, such as:
Does the tool use de-identification to remove personal identifiable information (PII), especially from training datasets and outputs?
Where is training data and fine-tuning data stored, copied and processed?
How does the vendor review and test accuracy and bias, and on what cadence?
Is there a way to opt in or out of data collection?
Where possible, implement explainability for AI and human review processes. Build trust and the business value of the AI by understanding the model and datasets enough to explain why the AI returned a given output.
Open-source software’s security benefits simultaneously pose security risks. Many open-source models can be deployed in your environment, giving you the benefit of your security controls. However, open-source models can expose the unsuspecting to new threats, including manipulation of outputs and harmful content by bad actors.
AI tech startups offering tools built on open AI can lack adequate cyber security, security teams, or secure development and maintenance practices. Organizations evaluating these vendors should ask targeted questions, such as:
Risk Leaders: Follow These 4 Strategies When Transitioning To Continuous Risk Management
Does the open project address cybersecurity issues?
Are the developers involved in the project demonstrating secure practices like those outlined by OWASP?
Have vulnerabilities and bugs been promptly remediated by the community?
Enterprises experimenting with AI tooling should continue following internal policies, processes, standards, and legal requirements. Consider best security practices like:
The tool’s source code should remain subject to vulnerability scanning.
Enable branch protection for AI integrations.
Interconnections should be encrypted in transit and databases at rest.
Establish boundary protection for the architecture and use cases.
A strong security posture will serve enterprises well in their AI explorations.
Integration and performance of AI tooling matters for both internal and external use cases at an organization.
Integration can affect many internal elements, like data pipelines, other models and analytics tools, increasing risk exposure and hampering product performance. Tools can also introduce dependencies upon integration, such as open source vector databases supporting model functionality. Consider how those elements affect your tool integration and use cases, and determine what additional adjustments are needed.
After integration, monitor AI’s impact on system performance. AI vendors may not carry a performance warranty, causing your organization to handle development if open-source AI does not meet your expectations. The costs associated with maintaining and scaling AI functions, including data cleaning and subject matter expertise time, climb quickly.
Open-source AI tooling offers enterprises an accessible and affordable way to accelerate innovation. Still, successful implementation requires scrutiny and a proactive compliance and security posture. An intentional evaluation strategy for hidden costs and considerations of leveraging open-source AI will ensure ethical and intelligent use.
Jessica Hill
Principal Legal Counsel, Product and Privacy, New Relic
Jessica Hill is an experienced product and privacy attorney adept at navigating the intersection of law and technology. She had been at New Relic for over three years, where she specializes in cross-functional initiatives to drive compliance and innovation.
Supercharge ITSM: How to Automate & Elevate Your IT Service Desk
2022 State of ITOps and SecOps
You May Also Like
Jan 17, 2025
Jan 16, 2025
Jan 7, 2025
Nov 20, 2024
Jan 21, 2025
Supercharge ITSM: How to Automate & Elevate Your IT Service Desk
Bajaj Allianz Life Insurance Case Study
Evaluation Guide: How to Choose a Network Monitoring Tool
Attacker Economics
VeloCloud SD-WAN Cloud Gateway Advantages
ITSM Vendor Rankings: See How Top Platforms Compare
Copyright © 2025. This website is owned and operated by Informa TechTarget, part of a global network that informs, influences and connects the world’s technology buyers and sellers. All copyright resides with them. Informa PLC’s registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. TechTarget, Inc.’s registered office is 275 Grove St. Newton, MA 02466.
source
This article was autogenerated from a news feed from CDO TIMES selected high quality news and research sources. There was no editorial review conducted beyond that by CDO TIMES staff. Need help with any of the topics in our articles? Schedule your free CDO TIMES Tech Navigator call today to stay ahead of the curve and gain insider advantages to propel your business!
Together, we power an unparalleled network of 220+ online properties covering 10,000+ granular topics, serving an audience of 50+ million professionals with original, objective content from trusted sources. We help you gain critical insights and make more informed decisions across your business priorities.
Open-Source AI has a raised profile due to its growth in exposure and popularity, but it is not without its own procedures, challenges, and idiosyncrasies governing its adoption and use.
February 28, 2025
Open-source AI projects are exploding in popularity and are contributing to PwC’s estimated $15.7 trillion impact AI will have on the global economy by 2030. However, some enterprises have hesitated to fully embrace AI.
In 2023, VentureBeat found that while more than 70% of companies were experimenting with AI, only 20% were willing and able to invest more.
Open-source tooling offers enterprises cost-effective, accessible AI use with benefits including customization, transparency and platform independence. But it also carries potentially hefty costs for the unprepared. As enterprises expand their AI experimentation, managing these risks becomes critical.
Many AI tools rely on vast stores of training data to develop models and generate outputs. For example, OpenAI’s GPT-3.5 was reportedly trained on 570 gigabytes of online text data, approximating 300 billion words.
More advanced models require even larger and often less transparent datasets. Some open-source AI tools are released without dataset disclosures or with overwhelming disclosures, limiting useful model evaluations and posing potential risks. For example, a code generation AI tool could be trained on proprietary, licensed datasets without permission, leading to unlicensed output, and potential liability.
What Netflix’s ‘Zero Day’ Got Right (and Wrong) About Cyberattacks
Open-source AI tools using open datasets still face challenges, such as evaluating data quality to ensure a dataset hasn't been corrupted, is regularly maintained, and includes data suited for the tool’s intended purpose.
Regardless of the data’s origins, enterprises should carefully review training data sources and tailor future datasets to the use case, where possible.
Proper data, model, and output licensing presents complicated issues for AI proliferation. The open-source community has been discussing the suitability of traditional open-source software licenses for AI models.
Current licensing ranges from freely open to partial use restrictions, but unclear criteria for qualifying as “open source” can lead to licensing confusion. The licensing question can trickle downstream: If a model produces output from a source with a viral license, you may need to adhere to that license’s requirements.
With models and datasets evolving constantly, evaluate every AI tool’s licensing against your chosen use case. Legal teams should help you understand limitations, restrictions and other requirements, like attribution or a flow-down of terms.
How to Overcome the Quantum Threat
As global AI regulations emerge and discussions swirl around the misuse of open-source models, companies should assess regulatory and privacy concerns for AI tech stacks.
At this stage, be comprehensive in your risk assessments. Ask AI vendors targeted questions, such as:
Does the tool use de-identification to remove personal identifiable information (PII), especially from training datasets and outputs?
Where is training data and fine-tuning data stored, copied and processed?
How does the vendor review and test accuracy and bias, and on what cadence?
Is there a way to opt in or out of data collection?
Where possible, implement explainability for AI and human review processes. Build trust and the business value of the AI by understanding the model and datasets enough to explain why the AI returned a given output.
Open-source software’s security benefits simultaneously pose security risks. Many open-source models can be deployed in your environment, giving you the benefit of your security controls. However, open-source models can expose the unsuspecting to new threats, including manipulation of outputs and harmful content by bad actors.
AI tech startups offering tools built on open AI can lack adequate cyber security, security teams, or secure development and maintenance practices. Organizations evaluating these vendors should ask targeted questions, such as:
Risk Leaders: Follow These 4 Strategies When Transitioning To Continuous Risk Management
Does the open project address cybersecurity issues?
Are the developers involved in the project demonstrating secure practices like those outlined by OWASP?
Have vulnerabilities and bugs been promptly remediated by the community?
Enterprises experimenting with AI tooling should continue following internal policies, processes, standards, and legal requirements. Consider best security practices like:
The tool’s source code should remain subject to vulnerability scanning.
Enable branch protection for AI integrations.
Interconnections should be encrypted in transit and databases at rest.
Establish boundary protection for the architecture and use cases.
A strong security posture will serve enterprises well in their AI explorations.
Integration and performance of AI tooling matters for both internal and external use cases at an organization.
Integration can affect many internal elements, like data pipelines, other models and analytics tools, increasing risk exposure and hampering product performance. Tools can also introduce dependencies upon integration, such as open source vector databases supporting model functionality. Consider how those elements affect your tool integration and use cases, and determine what additional adjustments are needed.
After integration, monitor AI’s impact on system performance. AI vendors may not carry a performance warranty, causing your organization to handle development if open-source AI does not meet your expectations. The costs associated with maintaining and scaling AI functions, including data cleaning and subject matter expertise time, climb quickly.
Open-source AI tooling offers enterprises an accessible and affordable way to accelerate innovation. Still, successful implementation requires scrutiny and a proactive compliance and security posture. An intentional evaluation strategy for hidden costs and considerations of leveraging open-source AI will ensure ethical and intelligent use.
Jessica Hill
Principal Legal Counsel, Product and Privacy, New Relic
Jessica Hill is an experienced product and privacy attorney adept at navigating the intersection of law and technology. She had been at New Relic for over three years, where she specializes in cross-functional initiatives to drive compliance and innovation.
Supercharge ITSM: How to Automate & Elevate Your IT Service Desk
2022 State of ITOps and SecOps
You May Also Like
Jan 17, 2025
Jan 16, 2025
Jan 7, 2025
Nov 20, 2024
Jan 21, 2025
Supercharge ITSM: How to Automate & Elevate Your IT Service Desk
Bajaj Allianz Life Insurance Case Study
Evaluation Guide: How to Choose a Network Monitoring Tool
Attacker Economics
VeloCloud SD-WAN Cloud Gateway Advantages
ITSM Vendor Rankings: See How Top Platforms Compare
Copyright © 2025. This website is owned and operated by Informa TechTarget, part of a global network that informs, influences and connects the world’s technology buyers and sellers. All copyright resides with them. Informa PLC’s registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. TechTarget, Inc.’s registered office is 275 Grove St. Newton, MA 02466.
source
This article was autogenerated from a news feed from CDO TIMES selected high quality news and research sources. There was no editorial review conducted beyond that by CDO TIMES staff. Need help with any of the topics in our articles? Schedule your free CDO TIMES Tech Navigator call today to stay ahead of the curve and gain insider advantages to propel your business!
