EmbedThis

Appearance

AI at the Edge ​

Artificial Intelligence (AI), particularly through Large Language Models (LLMs), significantly enhances edge devices by enabling more intelligent, autonomous operations directly at the point of use.

Ioto provides an intuitive AI library that simplifies interactions with cloud-based LLMs, facilitating tasks such as data classification, sensor data interpretation, information extraction, and logical reasoning. This capability is particularly beneficial for applications in predictive maintenance within industrial settings, smart agriculture, healthcare, smart homes, and environmental monitoring.

Developers typically select from three principal AI integration patterns: on-device models, cloud-based models, and hybrid models.

On-device language models operate entirely within the local hardware environment. This approach guarantees superior data privacy, reduced latency, and consistent operation regardless of network conditions, making it ideal for real-time applications with intermittent connectivity or stringent privacy requirements. However, the complexity and scale of these models are constrained by the limited computational resources of edge devices.

Cloud-based language models offload computationally intensive processing to remote servers, enabling the use of robust, large-scale LLMs that surpass the resource capabilities of edge devices. This design provides advanced features, seamless scalability, and simplified updates. Nevertheless, it relies on continuous internet connectivity and may introduce latency.

Hybrid approaches combine on-device processing with cloud-based computing. In this pattern, tasks that are privacy-sensitive or critical are executed locally, while more complex, resource-intensive operations are handled in the cloud. This approach effectively blends the strengths of privacy, responsiveness, and scalability, delivering an optimal balance that significantly improves system performance and user experience.

The Ioto AI library facilitates seamless interaction with cloud-based language models through integration with the OpenAI Chat Completion, Response, and Real-Time APIs.

OpenAI and Foundation Models ​

The Ioto AI provides support for the standard OpenAI Chat Completions API which is supported by many foundation models. While the core API is targeted at the OpenAI implementation, it is easily adapted for other models that utilize the same APIs.

Ioto AI APIs ​

Ioto provides support for the following LLM APIs:

Configuration ​

To enable the AI API, ensure the services.ai is set to true. The services.ai is a compilation and run-time control to initialize the AI library.

json5
services: {
    ai: true,
    ...
},

Then configure the OpenAI access key for your OpenAI account and define your preferred model.

json5
ai: {
    enable: true,
    provider: "openai",
    model: "gpt-4o",
    endpoint: "https://api.openai.com/v1",
    key: "sk-proj-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}

The ai.enable setting enables run-time control of the AI service.

If you are using another foundation LLM other than OpenAI, you can define the API endpoint for that service via the endpoint property.

APIs ​

References ​

Consult the OpenAI documentation for API details: