1. Introduction

With the advancement of language models and increasing computational demands, AI solutions have become indispensable in daily work routines. To ensure AI efficiency, robust computing resources are required. In many cases, this involves:

• High-performance hardware: Powerful processors, RAM, and, in certain situations, specialized graphics cards.
• High energy consumption: Running complex models can require a considerable amount of electricity, leading to higher bills and, in some cases, even exceeding the cost of a human intervention to solve a single problem.

2. Hardware and Electricity Costs in Practice

On my server, it is possible to monitor energy consumption while running AI tasks locally. In the presented example, the turbostat command was used to measure power usage in watts while the Llama model processed a query.

Monitoring Example:

sudo turbostat --show PkgWatt --interval 5
...
PkgWatt
1.83
1.83
...
PkgWatt
43.23
43.24
...
PkgWatt
1.82
1.82
...

The log shows that power consumption varies significantly during operation. Initially, the energy usage may be low, but as processing intensifies—such as with a complex query—consumption rises significantly, reaching high levels (e.g., 43 watts). This variation highlights that, for each task, energy costs can exceed expectations, especially if the AI requires multiple attempts and adjustments to generate the correct answer.

3. AI as an Essential Work Tool

Despite the high costs, AI has become an indispensable tool in today’s world. Much like a power tool, AI is designed to accelerate processes and enhance productivity. However, it is crucial to be aware of the associated investments:

• Hardware expenses: The acquisition and maintenance of powerful servers.
• Electricity bills: Especially for operations requiring intensive and continuous processing.
• Cost of errors and retries: During model training or execution, repeated cycles of trial and error further increase resource consumption.

4. Local Fine-Tuning: Customizing AI for Your Business

One major advantage of running AI locally is the ability to perform fine-tuning—adjusting the model for specific functions with sensitive company data. This process allows AI to:

• Be more accurate: Adapting better to the business’s unique needs.
• Enhance security: Keeping data internally reduces exposure risks.
• Optimize costs: A tailored model operates more efficiently, avoiding unnecessary energy and resource consumption.

Fine-tuning involves adjusting the language model’s parameters based on a specific dataset, allowing AI to better understand context and provide more precise responses. This customization is crucial for companies seeking a tailored solution, even if it requires higher investment in hardware and energy.

5. Private AIs and Token Costs

Beyond hardware and electricity expenses, it is essential to consider the costs associated with private AI solutions, such as ChatGPT. These services, often based on subscription models or token consumption, can become costly, especially in high-performance scenarios.

Reports suggest that OpenAI plans to charge around $20,000 per month to provide AI agents with PhD-level capabilities, demonstrating how costs can rise significantly as AI quality and performance improve.

In this context, companies must weigh the decision between developing in-house AI solutions—such as the case study with a local server—or relying on private services that, while offering advanced functionalities and continuous support, come with high costs due to token usage fees and premium services.

Thus, the choice between a local or private AI requires a thorough analysis of total costs, specific needs, and required performance levels.

6. Conclusion

In the coming years, we will see the rise of various applications—some with fine-tuned AI integration, others without. The trend suggests that as AI tools become more widespread, the need for optimization to reduce operational costs will also increase. While AI continues to prove indispensable, understanding the investments involved is crucial to balancing performance and resource consumption.

This case study, using a local server and wattage monitoring, illustrates how a single query can cause significant fluctuations in energy consumption. It demonstrates that even though AI is a revolutionary tool, careful planning is essential to ensure its benefits outweigh the operational costs.