AI gamble must be smart, not just fast – The Express Tribune
US
No matter how lucrative AI-driven public sector may be, these models must be thoroughly tested before being deployed
The future of data sharing changed drastically when the US realised that 9/11 was a failure of intelligence agencies to act in concert on then-available data and hence called the incident a "data fusion" crisis. The US Department of Homeland Security began setting up a robust network of "fusion centres" – state and locally run organisations that allow real-time sharing of critical intelligence and datasets between two or more government units for identifying red flags.
Fast forward to 2025, and now Artificial Intelligence (AI) is taking over such "fusion centres" worldwide – with possibilities that are endless. AI agents are replacing humans, and language models are generating insights that were previously unheard of. However, as is the case with every technology, the use of AI, especially in the public sector and in legal matters, remains a double-edged sword and must be handled with a pinch of salt.
For instance, in June 2023, Schwartz, an attorney with Levidow, Levidow & Oberman in New York, used ChatGPT for legal case research and was fined by the judge for citing false precedents with bogus names in his brief. The large language model (LLM) was apparently hallucinating – a problem where these chatbots make up fictitious data on their own.
Similarly, in March 2024, the Microsoft-powered chatbot MyCity gave incorrect legal information that could have led prospective businessmen to break the law. It falsely claimed that landlords could openly discriminate based on the income of tenants and that restaurant owners could take a share of their workers' tips.
Hence, when it comes to using AI, public institutions are now faced with a tough choice: should they rely on public AI models hosted by third parties such as ChatGPT, adopt open-source models such as LLaMA, or train their own proprietary AI models in the long run? Choosing the right AI strategy is crucial here.
In 2024, Air Canada's virtual assistant was found to be giving factually incorrect information about discounts to a customer who then took the matter to court and was awarded damages.
Similarly, when Denmark rolled out AI algorithms in its social security system, the system was found to have an inherent bias against marginalised groups such as the elderly, low-income families, migrants, and foreigners. Ninety per cent of the cases that AI marked as fraud later turned out to be genuine, and the whole episode is now taught as a classic case study in discrimination and breach of the European Union's (EU) AI Act's regulations on social scoring systems.
Therefore, if any public sector organisation chooses to use a third-party model trained by OpenAI in its operations, there is a risk of bias against people of colour and disadvantaged groups – as the training data scraped from the internet, social media and discussion forums is usually biased itself.
A good AI strategy involves thoughtful and controlled phased deployments with well-planned use cases. For example, the Department of Homeland Security (DHS) began with publicly available AI tools to improve employee productivity but also rolled out its AI vision and development roadmap. In the meantime, it focused on developing specialised AI applications – such as one to train officers dealing with asylum applications and conducting security investigations.
By December 2024, DHS had launched DHSChat on its internal secure network – a cutting-edge algorithm that can draft reports, streamline tasks, develop software, and, unlike other large language models, ensures employee data is protected and not used to train external models. In fact, as a best practice and as mandated by the Trump administration's executive order, DHS actively maintains its AI inventory, which includes a list of use cases related to AI in its operations.
For countries like Pakistan, our institutions could use a mix of public, open-source and proprietary models – depending on the nature of the task at hand. When it comes to using AI as the new Google, public models are usually fine, but for drafting memos and summarising reports, it is not advisable to use a public model. For that, the Ministry of IT or other institutions can host their own open-source AI models in their data centres or fine-tune them to develop proprietary models.
For critical systems, it is always recommended not to entirely replace existing automation with AI. There is a need to install a supervisor for fact-checking and verifying the output of AI models for hallucinations and bias. No matter how lucrative the idea of an AI-driven public sector may be, it is important to thoroughly test and check the behaviour of these models before deploying them.
The AI-based transformation project currently being executed at the Federal Board of Revenue (FBR) will serve as a test case for other AI-aspiring public agencies.
The writer is a Cambridge graduate and is working as a strategy consultant
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ
Updated May 07, 2025
Industry leaders urge unified response to tackle repressive tax measure, save industries
Updated May 07, 2025
.
Updated May 07, 2025
Urges tariff restructuring, simplified tax regime to boost industrial growth
Updated May 07, 2025
.
Updated May 07, 2025
Mulling limits on SEZs’ tax-free status from FY2025-26
Pakistan's governance crisis impedes prosperity
An ideal five-year economic policy
AI gamble must be smart, not just fast
A case to abolish capital gains tax
Pakistan's moment for structural reforms
Pakistan must navigate carefully
MrBeast's friend wins $417K after losing 100 pounds in 6 months amid coach’s death
Why China and Russia will not enter Iran-Israel war
US launches bombing of Iran’s three nuclear facilities
Signal-free corridor takes shape in Pindi
Body of veteran actress found days after her death
Heavy rains, thunderstorms predicted in Northern Pakistan
The four-day Indo-Pak spat
Iran-Israel conflict: what it means for Pakistan
PTMs: student perspective
Freedom from eternal Hindu scorn? Paradox of Jinnah's Pakistan
The economy of ignorance
Why China and Russia will not enter Iran-Israel war
This material may not be published, broadcast, rewritten, redistributed or derived from. Unless otherwise stated, all content is copyrighted © 2025 The Express Tribune.
source
This article was autogenerated from a news feed from CDO TIMES selected high quality news and research sources. There was no editorial review conducted beyond that by CDO TIMES staff. Need help with any of the topics in our articles? Schedule your free CDO TIMES Tech Navigator call today to stay ahead of the curve and gain insider advantages to propel your business!
No matter how lucrative AI-driven public sector may be, these models must be thoroughly tested before being deployed
The future of data sharing changed drastically when the US realised that 9/11 was a failure of intelligence agencies to act in concert on then-available data and hence called the incident a "data fusion" crisis. The US Department of Homeland Security began setting up a robust network of "fusion centres" – state and locally run organisations that allow real-time sharing of critical intelligence and datasets between two or more government units for identifying red flags.
Fast forward to 2025, and now Artificial Intelligence (AI) is taking over such "fusion centres" worldwide – with possibilities that are endless. AI agents are replacing humans, and language models are generating insights that were previously unheard of. However, as is the case with every technology, the use of AI, especially in the public sector and in legal matters, remains a double-edged sword and must be handled with a pinch of salt.
For instance, in June 2023, Schwartz, an attorney with Levidow, Levidow & Oberman in New York, used ChatGPT for legal case research and was fined by the judge for citing false precedents with bogus names in his brief. The large language model (LLM) was apparently hallucinating – a problem where these chatbots make up fictitious data on their own.
Similarly, in March 2024, the Microsoft-powered chatbot MyCity gave incorrect legal information that could have led prospective businessmen to break the law. It falsely claimed that landlords could openly discriminate based on the income of tenants and that restaurant owners could take a share of their workers' tips.
Hence, when it comes to using AI, public institutions are now faced with a tough choice: should they rely on public AI models hosted by third parties such as ChatGPT, adopt open-source models such as LLaMA, or train their own proprietary AI models in the long run? Choosing the right AI strategy is crucial here.
In 2024, Air Canada's virtual assistant was found to be giving factually incorrect information about discounts to a customer who then took the matter to court and was awarded damages.
Similarly, when Denmark rolled out AI algorithms in its social security system, the system was found to have an inherent bias against marginalised groups such as the elderly, low-income families, migrants, and foreigners. Ninety per cent of the cases that AI marked as fraud later turned out to be genuine, and the whole episode is now taught as a classic case study in discrimination and breach of the European Union's (EU) AI Act's regulations on social scoring systems.
Therefore, if any public sector organisation chooses to use a third-party model trained by OpenAI in its operations, there is a risk of bias against people of colour and disadvantaged groups – as the training data scraped from the internet, social media and discussion forums is usually biased itself.
A good AI strategy involves thoughtful and controlled phased deployments with well-planned use cases. For example, the Department of Homeland Security (DHS) began with publicly available AI tools to improve employee productivity but also rolled out its AI vision and development roadmap. In the meantime, it focused on developing specialised AI applications – such as one to train officers dealing with asylum applications and conducting security investigations.
By December 2024, DHS had launched DHSChat on its internal secure network – a cutting-edge algorithm that can draft reports, streamline tasks, develop software, and, unlike other large language models, ensures employee data is protected and not used to train external models. In fact, as a best practice and as mandated by the Trump administration's executive order, DHS actively maintains its AI inventory, which includes a list of use cases related to AI in its operations.
For countries like Pakistan, our institutions could use a mix of public, open-source and proprietary models – depending on the nature of the task at hand. When it comes to using AI as the new Google, public models are usually fine, but for drafting memos and summarising reports, it is not advisable to use a public model. For that, the Ministry of IT or other institutions can host their own open-source AI models in their data centres or fine-tune them to develop proprietary models.
For critical systems, it is always recommended not to entirely replace existing automation with AI. There is a need to install a supervisor for fact-checking and verifying the output of AI models for hallucinations and bias. No matter how lucrative the idea of an AI-driven public sector may be, it is important to thoroughly test and check the behaviour of these models before deploying them.
The AI-based transformation project currently being executed at the Federal Board of Revenue (FBR) will serve as a test case for other AI-aspiring public agencies.
The writer is a Cambridge graduate and is working as a strategy consultant
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ
Updated May 07, 2025
Industry leaders urge unified response to tackle repressive tax measure, save industries
Updated May 07, 2025
.
Updated May 07, 2025
Urges tariff restructuring, simplified tax regime to boost industrial growth
Updated May 07, 2025
.
Updated May 07, 2025
Mulling limits on SEZs’ tax-free status from FY2025-26
Pakistan's governance crisis impedes prosperity
An ideal five-year economic policy
AI gamble must be smart, not just fast
A case to abolish capital gains tax
Pakistan's moment for structural reforms
Pakistan must navigate carefully
MrBeast's friend wins $417K after losing 100 pounds in 6 months amid coach’s death
Why China and Russia will not enter Iran-Israel war
US launches bombing of Iran’s three nuclear facilities
Signal-free corridor takes shape in Pindi
Body of veteran actress found days after her death
Heavy rains, thunderstorms predicted in Northern Pakistan
The four-day Indo-Pak spat
Iran-Israel conflict: what it means for Pakistan
PTMs: student perspective
Freedom from eternal Hindu scorn? Paradox of Jinnah's Pakistan
The economy of ignorance
Why China and Russia will not enter Iran-Israel war
This material may not be published, broadcast, rewritten, redistributed or derived from. Unless otherwise stated, all content is copyrighted © 2025 The Express Tribune.
source
This article was autogenerated from a news feed from CDO TIMES selected high quality news and research sources. There was no editorial review conducted beyond that by CDO TIMES staff. Need help with any of the topics in our articles? Schedule your free CDO TIMES Tech Navigator call today to stay ahead of the curve and gain insider advantages to propel your business!
