AI Agent text to technical drawing analysis

The client

The customer is a distinguished multinational enterprise specializing in the design, manufacturing, and commercialization of light, medium, and heavy-duty commercial vehicles—including trucks, vans, and buses.

The challenge

The challenge was to develop an AI agent capable of extracting measurements from a large dataset of technical drawings using concise prompts that specify attributes such as the requested measurement, the component, and the referenced model. Additionally, the agent must be able to identify potential measurement inconsistencies across different drawing revisions. The AI agent must be capable of handling multiple concurrent requests from technical users worldwide, pertaining to the customer’s operations. Furthermore, the agent operates on a dataset on the order of thousands of technical drawings, and the system is designed to scale automatically.

The solution

The idea was to develop an AI agent capable of extracting measurements from a large dataset of technical drawings using concise prompts that specify attributes such as the requested measurement, the component, and the referenced model. Additionally, the agent must be able to identify potential measurement inconsistencies across different drawing revisions.

The AI agent must be capable of handling multiple concurrent requests from technical users worldwide, pertaining to the customer’s operations. Furthermore, the agent operates on a dataset on the order of thousands of technical drawings, and the system is designed to scale automatically.

The designated IDE-like environment was Amazon SageMaker AI. The solution leverages multiple AWS services, including Amazon Bedrock for LLM API calls, and Amazon Textract and Amazon Rekognition for OCR tasks. Amazon Cognito was used as the Single Sign ON (SSO) provider.

Additional AWS cloud components used:

  • SageMaker AI
  • BedRock
  • Textract
  • Rekognition
  • S3
  • DynamoDB
  • Lambda
  • API gateway

Beyond the authentication required to access the service, the architecture is serverless to avoid persistent storage of model inputs and outputs; additionally, the design prevents sharing data with third-party providers.

Model precision exceeds all customer-defined thresholds. The end-to-end pipeline was scaled to reduce response time from minutes to seconds, enabling near-real-time delivery of requested information.

Key performance metrics captured include request success rate (percentage of responded requests) and accuracy rate (percentage of correctly answered requests).

Workload resilience is achieved via a multi-region deployment.

Architecture for serverlss processing of tech drawings

The Outcome

The project was co-developed with the customer’s team across all phases. Joint R&D activities focused on the application of novel AI technologies followed the same collaborative model.

AI capabilities can enable any organization that relies on technical documentation—ranging from plain text to complex engineering drawings—to automate management and review workflows, thereby substantially enhancing the quality, consistency, and maintainability of its technical materials.