How to Build an AIoT System: Architecture, Edge AI, and Real Use Cases

IoT devices generate enormous amounts of data, but much of it is never analyzed in time to support decisions. Understanding how to build an AIoT system starts with connecting device data to models that can detect anomalies, predict failures, and trigger actions.

Building on the fundamentals of how to develop an IoT application, AIoT adds intelligent data processing at the edge or in the cloud. This guide explains how AI in IoT works, covering system architecture, model deployment, security, and practical development steps, with real-world examples to help teams plan their first AIoT project.

Key Takeaways

• AIoT combines IoT and AI: Connected devices collect data, while AI models turn it into predictions, alerts, and automated decisions.
• Model placement matters: Edge, cloud, and hybrid architectures offer different trade-offs in latency, connectivity, cost, and data privacy.
• Start with a pilot: One measurable use case and real device data provide a practical basis for testing performance before scaling.
• Security is a legal concern: The UK’s PSTI regulations already apply to relevant connected consumer products, while the EU Cyber Resilience Act is being phased in.

What Is AIoT?

AIoT (Artificial Intelligence of Things) combines Internet of Things (IoT) devices with AI models so connected systems can collect data, analyze it, and take action. Unlike traditional IoT, which primarily monitors conditions and sends data to other systems, AIoT can identify patterns, predict problems, and make decisions automatically.

The AIoT meaning is simple: the Internet of Things and AI work as one system, where devices supply data and models turn it into decisions.

In other words, artificial intelligence and IoT work as one system: IoT supplies the data, and AI turns it into decisions. For example, a factory sensor may continuously measure equipment vibration. In a traditional IoT setup, it sends readings to a dashboard for monitoring. With AIoT, a model can detect early signs of mechanical failure and automatically generate a maintenance request.

The table below compares traditional IoT and AIoT in terms of data processing, decision-making, and response.

AIoT does not eliminate the need for cloud infrastructure or human oversight. Instead, it allows systems to process data where it makes the most sense, based on response time, connectivity, and operational requirements.

How AI and IoT Work Together: The 4 Layers of an AIoT Architecture

When the Internet of Things and AI are combined, an AIoT architecture connects physical devices, AI processing, data infrastructure, and business applications.  Each layer has a specific role, from collecting sensor readings to turning predictions into actions. Although the components vary by project, most AIoT systems can be organized into four layers.

1. Devices and Sensors

The device layer collects data from physical environments through temperature sensors, vibration monitors, cameras, GPS modules, and other connected equipment. Microcontrollers such as STM32 and ESP32 can preprocess readings or run lightweight AI models directly on the device.

Data quality starts here. Incorrect sensor calibration, inconsistent sampling intervals, or missing timestamps can affect model accuracy. Devices also need secure identities and reliable communication methods to transmit readings to gateways or backend systems.

2. Edge and Gateways

Edge gateways receive data from one or more devices and process it close to the source. They can run AI inference, filter unnecessary readings, buffer data during connectivity interruptions, and translate industrial protocols such as Modbus or OPC UA into MQTT or other formats supported by the platform.

For time-sensitive applications, local inference allows equipment alerts to be generated without waiting for a cloud response. Gateways can forward relevant events and aggregated measurements once connectivity is available.

3. Cloud or On-Prem Platform

The platform layer handles workloads that require more computing power, shared infrastructure, or long-term data access. Typical components include time-series databases, data pipelines, model training environments, model registries, and tools for monitoring connected device fleets.

Cloud infrastructure offers flexible computing resources, while on-premises deployment can help organizations keep operational data within their own networks. Both approaches can support model versioning, deployment management, and analysis of data collected across multiple locations.

4. Applications and Integrations

The application layer makes AIoT insights available to the people and systems responsible for acting on them. Dashboards display equipment conditions, mobile apps notify technicians, and integrations connect predictions to enterprise resource planning (ERP), computerized maintenance management systems (CMMS), or field service management (FSM) platforms.

For example, an anomaly detected at the edge can raise an alert, appear on a maintenance dashboard, and create a work order. This connection between AI predictions and operational workflows is what makes the architecture useful beyond data collection.

Edge AI vs Cloud AI: Where Should the Model Run?

AIoT edge computing moves inference closer to the sensors, which reduces response time and network dependency. One of the most important decisions in AIoT architecture is where AI inference happens. Running a model directly on a device reduces network dependency and response time, while cloud processing provides more computing resources for complex models and large-scale analytics.

The choice depends on the task, hardware limitations, connectivity, and response time requirements. Edge AI can run on smartphones, gateways, industrial computers, or local servers. In industrial IoT systems, local processing is particularly useful when equipment needs to operate despite network interruptions.

AIoT Hardware and Deployment Options

The table compares five deployment options, from low-power sensors to cloud infrastructure.

Memory values are illustrative hardware configurations, not minimum model requirements. Actual capacity depends on model size, precision, runtime, and concurrent workloads.

For example, Google’s LiteRT for Microcontrollers (formerly TensorFlow Lite for Microcontrollers) has a core runtime that fits in 16 KB on an Arm Cortex-M3 and can run many basic models. The model and working memory need additional space. NVIDIA Jetson Orin modules provide configurations from 4 GB to 64 GB for more demanding edge workloads.

Five Criteria for Choosing Where AI Runs

The right deployment location becomes clearer when teams evaluate five practical constraints:

1. Latency: Safety alerts and machine-control decisions may require local inference to avoid network delays.
2. Connectivity: Devices operating with intermittent internet access need offline processing or local fallback behavior.
3. Power budget: Battery-powered sensors require smaller models and fewer computations than mains-powered gateways.
4. Privacy: Sensitive video, audio, or operational data may need to remain on the device or within a private network.
5. Cost per device: Hardware, cloud inference, bandwidth, maintenance, and updates all affect total ownership costs.

Model size is particularly important for generative AI. Running on-device LLMs requires enough memory and processing capacity for model weights, inference, and application workloads.

Why Hybrid AIoT Architectures Often Make Sense

For many AIoT projects, a hybrid architecture offers a practical balance. Devices or gateways handle time-sensitive inference, while cloud services support model training, analytics, storage, and updates.

For example, in fleet management software, onboard devices can detect vehicle anomalies locally, even without internet access. Once connectivity returns, they send selected events to the cloud for fleet-wide analysis and maintenance planning.

The goal is not to run every model at the edge, but to place each workload where it meets the project’s response-time, reliability, privacy, and cost requirements.

“You can build an accurate AI model and still end up with a failed AIoT project. If the prediction arrives too late, gets lost during a network outage, or never reaches the person responsible for acting on it, accuracy means very little. The system has to work beyond the model.”

— Vitaly Tormanov, Head of Mobile Solutions, SCAND

How to Build an AIoT System in 7 Steps

Building an AIoT system requires more than connecting sensors to AI models. Devices, communication protocols, data pipelines, and business applications must work together. The following steps explain how to plan, develop, test, and scale an AIoT solution.

Step 1. Start With One Use Case and a Business Metric

The first step is to identify a business problem that AIoT can realistically solve. For example, a manufacturer might aim to reduce unplanned pump downtime by 20% through predictive maintenance. Historical maintenance records, equipment failures, and repair costs help establish a baseline for measuring progress.

The team then defines what the AI model should detect, how the system should respond, and which metrics will indicate success. These may include downtime, false alarm rates, maintenance costs, or energy consumption. The 20% reduction is an illustrative target, not a guaranteed outcome.

Common mistake: Starting with a broad goal such as “add AI to our factory” without a measurable business objective.

Step 2. Audit Your Devices and Data

Before choosing a model, assess the existing IoT infrastructure. Document sensor types, communication interfaces, sampling rates, device capabilities, and available data storage. Check whether the collected information is sufficient for the selected use case and identify any gaps in coverage.

Data quality is equally important. Missing readings, inconsistent timestamps, sensor drift, and insufficient failure records can affect prediction accuracy. When labeled failure data is limited, anomaly detection or additional data collection may be more practical than supervised learning. Data ownership and access requirements should also be established.

Common mistake: Selecting an AI model before confirming that suitable training data is available.

Step 3. Choose Hardware and Connectivity

Hardware selection depends on processing requirements, memory, power consumption, environmental conditions, and communication range. Existing sensors may only need an additional edge gateway rather than complete replacement. Microcontrollers suit lightweight processing, while gateways can collect data from multiple devices and run more demanding models.

Connectivity options serve different purposes. BLE supports nearby low-power devices, LoRaWAN and NB-IoT suit remote sensors, and Modbus or OPC UA connect industrial equipment. MQTT supports message exchange between connected systems. Reliable IoT device integration also requires authentication, protocol translation, and recovery from connectivity failures.

Common mistake: Selecting hardware or protocols without considering actual bandwidth, range, power, and integration requirements.

Step 4. Decide Where Each Model Runs

AI inference can run on microcontrollers, smartphones, edge gateways, local servers, or cloud infrastructure. The decision depends on latency, available computing resources, connectivity, privacy, and operating costs. Time-sensitive applications often benefit from local processing, while cloud systems provide resources for model training and large-scale analytics.

Many projects use a hybrid architecture. For example, a factory gateway can detect equipment anomalies locally, even during an internet outage, while cloud services analyze performance across multiple sites. The architecture should define which data stays on devices, what reaches the cloud, and how the system behaves when connectivity fails.

Common mistake: Sending all sensor data to the cloud by default, even when local inference would reduce latency and network dependency.

Step 5. Build and Optimize the Models

Model development starts with preparing sensor data, aligning timestamps, handling missing values, and extracting useful features. Depending on the application, the team may use anomaly detection, classification, forecasting, or computer vision. Evaluation should cover missed events, false alarms, and inference latency alongside prediction accuracy.

Models intended for constrained hardware may require quantization, pruning, or compression to reduce memory and processing requirements. Optimization can affect accuracy, so the final model must be tested on its target hardware. Real-device testing should measure inference speed, memory usage, energy consumption, and performance under realistic operating conditions.

Common mistake: Assuming a model that performs well during development will run equally well on an embedded device.

Step 6. Build the Platform, Apps, and Integrations

An AIoT platform connects devices, AI models, and users through telemetry ingestion, time-series storage, device management, APIs, and dashboards. Mobile applications can support equipment monitoring and configuration, and BLE app development covers direct communication with nearby connected devices.

Predictions become useful when they trigger operational actions. For example, a detected equipment fault can automatically generate a maintenance request in field service management software. The platform also needs access controls, encrypted communication, and reliable handling of delayed or duplicate messages.

Common mistake: Building accurate AI predictions without connecting them to applications where people can act on the results.

Step 7. Pilot, Then Scale With OTA Updates and Monitoring

Run the first release as a pilot on 10–50 devices, depending on equipment cost and operating conditions. Test normal operation, network outages, and device failures, and track business outcomes, prediction accuracy, and response times against the metric set in Step 1.

Scaling adds fleet-level work. Signed OTA updates, staged rollouts, and rollback procedures, supported by secure firmware development, let teams update devices and models without visiting each site. Monitoring should track prediction accuracy and model drift, since sensor behavior changes with wear, seasons, and new equipment.

Common mistake: Treating a successful pilot as a finished product without planning for updates, security maintenance, and ongoing monitoring.

If you’d rather not assemble this stack alone, our team provides AIoT development services from device firmware to cloud models.

AIoT Use Cases by Industry

AIoT applications follow one pattern: sensors collect data, a model finds the signal in it, and a system acts on the result. The AIoT examples below show where this works in practice and which data each case relies on.

Manufacturing: Predictive Maintenance and Visual Inspection

AIoT manufacturing projects typically start with predictive maintenance and visual inspection. Vibration, temperature, and current sensors feed models that flag early signs of bearing or motor wear before a breakdown. Cameras with computer vision detect surface defects and missing components on a gateway beside the line.

Logistics and Fleets

Telematics units report location, speed, engine data, and cargo temperature in the cold chain. Models detect harsh driving and developing vehicle faults, suggest routes based on traffic and delivery windows, and alert when a refrigerated trailer leaves its temperature range. Local processing keeps alerts working without coverage.

Field Service and Utilities

Pumps, meters, transformers, and HVAC units produce condition data long before they fail. A model trained on this history can predict a fault and open a work order with the asset, likely cause, and required parts attached. Predictions become scheduled visits as described in AI in field service management.

Agriculture

Soil moisture sensors, weather stations, and GPS-equipped sampling tools produce field data that AI can turn into irrigation schedules, sampling plans, and optimized routes. Because coverage in fields is unreliable, data is stored locally and synchronized later. SCAND’s soil sampling project, described below, applies this approach.

Smart Buildings and Retail

In smart buildings and AIoT smart home systems, occupancy, temperature, and energy data let models adjust heating, lighting, and ventilation to actual use. Among AIoT retail applications, shelf cameras and weight sensors detect empty shelves, and footfall sensors help forecast staffing. Video data usually stays on local devices.

Healthcare

AIoT healthcare applications center on remote patient monitoring. Wearables and home devices report heart rate, oxygen saturation, or glucose readings, and models flag trends that need clinical attention. Medical software is regulated, so requirements for the device and its AI features should be clarified at the start.

Generative AI and LLMs in IoT

Generative AI IoT applications go beyond detecting anomalies and predicting equipment failures. Large language models (LLMs) help people interact with connected devices, understand sensor data, and respond to operational problems through natural language.

Voice-controlled interfaces let operators check equipment status or adjust supported settings without navigating dashboards. A maintenance assistant can use retrieval-augmented generation (RAG) to search technical manuals and answer equipment-related questions. Choosing the best local LLM for RAG depends on hardware resources, document retrieval quality, and the required response time.

LLMs can also explain AIoT alerts in plain language, summarizing sensor readings and suggesting possible causes. With appropriate permissions and validation, an AI agent can create maintenance tickets or assign technicians automatically.

Where Should LLMs Run?

Cloud models provide substantial computing capacity, but sensitive operational data may need to remain within company infrastructure. A private LLM can run on a local server or a sufficiently powerful edge gateway, allowing organizations to control data access and processing.

Teams planning to run a local LLM must consider GPU memory, model size, response time, and concurrent requests. Smaller models may suit local assistants, while more demanding tasks require dedicated servers.

SCAND Example: Local LLM for Logistics

In a logistics project, SCAND runs a local LLM on NVIDIA RTX hardware that extracts transportation details from Telegram messages instead of calling external model APIs. It was not an IoT deployment, but it shows how a locally hosted model can handle high-volume operational data.

AIoT Security and Regulation

AIoT systems connect physical devices, networks, AI models, and cloud services, creating multiple potential entry points for attackers. A compromised sensor or gateway can expose sensitive data, disrupt operations, or send manipulated readings to AI models. Security therefore needs to cover the entire system, from device firmware to backend infrastructure.

A secure AIoT architecture should include:

• Device identity: Unique credentials or certificates for each device, with no universal default passwords.
• Secure communication: Encryption in transit, authenticated connections, and restricted access to device data.
• Firmware protection: Signed updates, secure boot where supported, and controlled OTA deployment.
• Device inventory: Records of firmware versions, installed updates, known vulnerabilities, and support status.
• Access control: Least-privilege permissions for devices, applications, and administrative accounts.

Regulation also affects AIoT product development. The UK’s Product Security and Telecommunications Infrastructure (PSTI) regime has applied since April 29, 2024. Relevant consumer connected products must not use universal default passwords. Manufacturers must also provide a vulnerability reporting channel and publish the minimum period during which security updates will be supplied.

In the EU, the Cyber Resilience Act (Regulation 2024/2847) introduces cybersecurity requirements for products with digital elements. Since September 11, 2026, manufacturers of covered products must report actively exploited vulnerabilities and severe security incidents, with an early warning generally required within 24 hours of becoming aware. The regulation’s main requirements apply from December 11, 2027.

Not every AIoT deployment falls under the same rules. Before release, teams should determine which products and markets are covered, document their security controls, and establish vulnerability response and update procedures.

Common Challenges and What Drives the Cost

AIoT development involves challenges that rarely appear in standard software projects. Teams must work with physical devices, unreliable networks, limited computing resources, and AI models that need continuous monitoring. These factors affect both development complexity and long-term costs.

Common AIoT Development Challenges

• Different devices and protocols: Sensors from different manufacturers may use incompatible interfaces and data formats, requiring custom adapters or middleware.
• Limited training data: Historical equipment failures are often poorly documented, making supervised learning difficult. Additional data collection or anomaly detection may be necessary.
• Hardware constraints: AI models that perform well on powerful servers may exceed the memory, processing, or power limits of edge devices.
• Unreliable connectivity: Network interruptions can delay data transmission and alerts. Local buffering, retry mechanisms, and offline processing help maintain operation.
• Updates across device fleets: Deploying firmware and models to thousands of devices requires secure OTA updates, version tracking, and rollback procedures.
• Security risks: Each connected device creates another potential attack surface. Authentication, encryption, patch management, and vulnerability monitoring must cover the entire system.

What Affects AIoT Development Cost?

The budget depends on several technical and operational decisions:

• Device diversity: More hardware models and communication protocols increase integration and testing effort.
• Custom hardware and firmware: Developing devices from scratch adds engineering, prototyping, and validation work.
• Edge vs. cloud processing: Local inference requires suitable hardware, while cloud processing brings ongoing computing and data transfer costs.
• Data volume: High-frequency sensor readings, video, and audio increase storage, processing, and network requirements.
• Business integrations: Connecting AIoT with ERP, CMMS, or field service platforms adds API development and workflow testing.
• Certification and compliance: Industry regulations and product certification may require additional testing and documentation.

Understanding these factors early helps teams estimate the full system rather than only its software components. The same considerations influence IoT app development cost, particularly when projects involve multiple device types, custom firmware, and cloud infrastructure.

How SCAND Builds AIoT Systems

SCAND combines experience in IoT integration, embedded software, mobile development, and AI. Our projects show how connected devices, offline data processing, and locally deployed models can address practical business problems.

AI-Assisted Soil Sampling and Field Management

In our soil sampling project, SCAND connected agricultural sampling equipment to a mobile application via BLE. The solution combines GPS data from devices and smartphones, offline storage, and AI-assisted route optimization. Field operations became up to 25% faster, with data synchronized when connectivity returned.

Connected Car and Fleet Monitoring

SCAND developed a connected car application that collects vehicle telemetry and supports remote control through BLE, Wi-Fi Direct, and cellular connections. A reusable hardware-control layer standardized communication with different devices, reducing the development time needed to integrate new hardware models by over 50%.

Local AI for Logistics

For a logistics automation project, SCAND deployed a local LLM on NVIDIA RTX hardware to extract transportation details from Telegram messages. The system processes 3,000–4,000 messages daily, demonstrating how locally hosted AI can handle operational workloads without relying on external model APIs.

These projects demonstrate capabilities relevant to AIoT development, from device communication and field applications to AI processing and integration with business systems.

Frequently Asked Questions (FAQs)

What is the difference between IoT and AIoT?

IoT systems connect physical devices to collect and exchange data, often for monitoring and remote control. AIoT adds artificial intelligence to analyze this data, recognize patterns, predict problems, and support automated decisions. For example, an IoT sensor reports equipment vibration, while an AIoT system can detect signs of failure and trigger a maintenance alert.

Can AI run directly on IoT devices?

Yes. Small AI models can run on microcontrollers, smart sensors, and other devices with limited computing resources. Technologies such as TinyML make local inference possible without sending every reading to the cloud. However, larger models may require an edge gateway, local server, or cloud infrastructure, depending on memory, processing power, and energy requirements.

What hardware do I need to build an AIoT system?

The hardware depends on the application and where AI processing takes place. Simple anomaly detection may run on microcontrollers, while computer vision often requires devices such as NVIDIA Jetson or industrial PCs. More demanding workloads may need GPU servers. Sensors, connectivity modules, gateways, and power requirements should also be considered when planning the system.

Do AIoT systems work without an internet connection?

Yes, if the architecture supports local processing. AI models running on devices or edge gateways can analyze sensor data and generate alerts without internet access. Data can be stored locally and synchronized when connectivity returns. However, cloud-dependent features, remote monitoring, and centralized analytics may remain unavailable until the connection is restored.

How long does it take to build an AIoT pilot?

An AIoT pilot’s timeline depends on the selected use case, available hardware, data quality, and integration requirements. Projects using existing sensors and historical data may require less preparation than those involving custom devices or new data collection. A useful pilot should test the complete workflow on real device data and measure defined business outcomes.

Do I need to replace my existing devices to add AI?

Not necessarily. Existing IoT devices can often support AIoT through an additional edge gateway or cloud-based analytics platform. In some cases, firmware updates allow devices to perform lightweight inference directly. The best approach depends on available memory, processing capacity, communication protocols, and whether the hardware can support secure updates.