Augmented Reality Vehicle Maintenance & Training System

Navy SBIR 25.2- Topic N252-076
Marine Corps Systems Command (MCSC)
Pre-release 4/2/25   Opens to accept proposals 4/23/25   Closes 5/21/25 12:00pm ET
   [ View Q&A ]

N252-076 TITLE: Augmented Reality Vehicle Maintenance & Training System

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Advanced Computing and Software;Human-Machine Interfaces;Sustainment

OBJECTIVE: Develop an Augmented Reality (AR) capability that aids a maintainer with replacing the brake pads, brake disk, fuel filter, and water separator on a Ultra Light Tactical Vehicle (ULTV). The AR capability shall provide a reach back function in which the video feed of the maintainer’s perspective can be wirelessly live streamed from one Electronics Maintenance Support System (EMSS) to another EMSS. The EMSS shall be the host system for the AR capability. The Prototype Device shall be designed with a platform agnostic architecture, in which the capability can be expanded to maintenance actions on multiple Marine Corps platforms.

DESCRIPTION: The problem at hand that the prototype device is intended to solve is a lack of ground vehicle maintenance training and overall maintenance experience. Vehicle maintainers in the Marine Corps have a wide range of varying experience, with some younger Marines having little to no experience with vehicle maintenance. To improve upon this knowledge gap, the Marine Corps is exploring AR to act as a real-time visual aid while conducting vehicle maintenance. The AR capability prototype device is intended to automatically detect vehicle components that are integral to the maintenance task at hand. The corresponding authoritative technical manuals and relevant schematics will be presented to the maintainer automatically upon component detection.

Another important objective of the AR capability prototype device is to provide a reach back capability in which an inexperienced Marine can contact a Subject Matter Expert (SME) in real-time. The end goal is to be able to have the SME be in an entirely different location than the maintainer, be able to see the live visual feed of the maintainer and provide resolution to the issue at-hand. This capability will directly support the Marine Corps approach of Expeditionary Advanced Base Operations (EABO).

The following is a detailed list of the scope of this SBIR effort:

      • Develop an AR capability that:
        • can visually aid the user with replacing the brake pads, brake disk, fuel filter, and water separator on a ULTV, using ULTV schematics and technical manuals as the authoritative source.
        • can replace all four ULTV components: brake pads, brake disk, fuel filter, and water separator (Threshold (T)); replace all four ULTV components listed in the threshold, as well as any additional maintenance actions beyond those four (Objective (O)).
        • integrate with the EMSS hardware and software configuration.
        • is capable of providing a live video feed from the AR device to a different EMSS via wireless transmission.
      • Augmented Reality Visual Aid Size:
        • Nominal hindrance to maintenance tasks while wearing a noticeable but not uncomfortable device (T); minimum hindrance to maintenance tasks while wearing a device with a small footprint and does not obscure the maintainer from the task at hand (O).
      • Augmented Reality Visual Aid Device:
        • Wired operation between the device and EMSS (T); wireless operation between the Prototype Device and EMSS (O).
      • Battery Life:
        • One-hour battery life while in operation (T); two-hour battery life while in operation (O).
      • System Weight:
        • Two person portable with all accessories (62 lbs.) in accordance with MIL-STD-1472H (T); one person portable with all accessories (31 lbs.) in accordance with MIL-STD-1472H (O).
      • Ruggedized:
        • Not ruggedized (T); ruggedized in accordance with MIL-STD-810H (O).

Note: The replacement cost should be considered during prototype device development. If the prototype device is not ruggedized, it is more likely to break and be in need of replacement.

PHASE I: Identify a technically capable solution for providing an AR capability within the hardware and software constraints of the EMSS. The required Phase I deliverables shall be a report which outlines the hardware and software approach of the AR capability. The report is expected to include the following:

      • Analysis of AR hardware solutions, with a definition of pros, cons, tradeoffs, and a recommended prototype hardware solution
      • Analysis of AR software solutions, with a focus on ease of development, software supportability, integration into EMSS, cyber compliance considerations, and future integration/expandability of the capability.
      • Analysis of existing ULTV models and schematics, with a determination on if the existing Technical Data Package is sufficient to produce a prototype solution. If the models provided are insufficient, the proposal shall include a concept for creating the models utilizing 3D scanning methods. A ULTV will be made available to the prototype developer to accomplish this effort.

PHASE II: Produce and validate an AR capability that can assist a maintainer with changing out the brake pads, brake disk, fuel filter, and water separator on a ULTV; and provides a wireless video feed reach back capability (O). The prototype device shall be capable of identifying components that are integral to performing the steps needed to change the ULTV brake pads, brake disk, fuel filter, and water separator. Once a component is identified, the prototype device shall visually display the corresponding Technical Manual and schematic information to the user.

PHASE III DUAL USE APPLICATIONS: Support the Marine Corps in transitioning the final product into its EMSS.

The commercial automative industry has started to integrate AR maintenance aids into their processes and practices. The government prototype device solution will be open source, which means that commercial automative industry companies that have not adopted this capability could utilize the prototype device solution.

This solution also could be utilized in the civilian environment for personal maintenance actions. The older generation is more knowledgeable on conducting car maintenance than the younger generation. This solution could be sold as a training aid to schools or vehicle hobbyists that need a way to teach and learn vehicle maintenance.

Additionally, this capability could have uses in different commercial sectors, such as electronics, home repair, plumbing, etc.

REFERENCES:

  1. Department of Defense. "MIL-STD-810H CHANGE 1, ENVIRONMENTAL ENGINEERING CONSIDERATIONS AND LABORATORY TESTS." 18 May 2022. https://cvgstrategy.com/wp-content/uploads/2023/04/MIL-STD-810H-change-1.pdf
  2. Department of Defense Design Criteria Standard. "MIL-STD-1472H, Human Engineering." 15 September 2020. https://quicksearch.dla.mil/Transient/3CC8027A50984AA6A6C65CBF14128E46.pdf
  3. Getac. "EMSS Laptop Hardware/Software Specification B360." https://www.getac.com/us/products/laptops/b360-pro/#tech-specs

KEYWORDS: Augmented; Reality; Training; Aid; Maintenance; Decision; Artificial Intelligence

** TOPIC NOTICE **

The Navy Topic above is an "unofficial" copy from the Navy Topics in the DoD 25.2 SBIR BAA. Please see the official DoD Topic website at www.dodsbirsttr.mil/submissions/solicitation-documents/active-solicitations for any updates.

The DoD issued its Navy 25.2 SBIR Topics pre-release on April 2, 2025 which opens to receive proposals on April 23, 2025, and closes May 21, 2025 (12:00pm ET).

Direct Contact with Topic Authors: During the pre-release period (April 2, 2025, through April 22, 2025) proposing firms have an opportunity to directly contact the Technical Point of Contact (TPOC) to ask technical questions about the specific BAA topic. The TPOC contact information is listed in each topic description. Once DoD begins accepting proposals on April 23, 2025 no further direct contact between proposers and topic authors is allowed unless the Topic Author is responding to a question submitted during the Pre-release period.

DoD On-line Q&A System: After the pre-release period, until May 7, 2025, at 12:00 PM ET, proposers may submit written questions through the DoD On-line Topic Q&A at https://www.dodsbirsttr.mil/submissions/login/ by logging in and following instructions. In the Topic Q&A system, the questioner and respondent remain anonymous but all questions and answers are posted for general viewing.

DoD Topics Search Tool: Visit the DoD Topic Search Tool at www.dodsbirsttr.mil/topics-app/ to find topics by keyword across all DoD Components participating in this BAA.

Help: If you have general questions about the DoD SBIR program, please contact the DoD SBIR Help Desk via email at DoDSBIRSupport@reisystems.com

Topic Q & A

5/7/25  Q. Upon award, what type of involvement does the Marine Corps (e.g., Technical Points of Contact, Marine Corps Systems Command, etc.) anticipate having during Phase 1, 2, and 3?
   A. Generally the MCSC TPOC will manage the effort through Phase I and Phase II.
5/5/25  Q. Question 1: Will the government confirm that if we choose to submit letters of support, we can place them in volume 5 rather than in volume 2?
Question 2: Will the government confirm that we can submit Resumes in volume 5 rather than in volume 2?
   A. As noted in the Navy SBIR 25.2 instruction, Supporting Documents (Volume 5) will not be considered in the selection process and will only undergo a compliance review to ensure the proposing small business concern has included items in accordance with the PHASE I SUBMISSION INSTRUCTIONS section...
A small business concern can insert Letter(s) of Support as an image to Volume 2 and resize that image, as desired.

It is allowable for a small business concern to include resumes in Volume 5, however, Volume 5 will not be considered in the selection process. Relevant qualifications for proposed Key Personnel will be included in Volume 2.
5/5/25  Q. Q1: Can we get a representative collection of Marine Corps Programs of Instruction in expected digital formats and a sample of existing legacy course materials.
Q2: Clarification on whether commercial LLMs can be directly used or development of in-house models is required to comply with data privacy standards.
   A.
  1. We plan to award to three vendors for Phase I. The three vendors selected will be provided an EMSS which will have all of the relevant technical manuals, schematics, drawings, etc.
  2. Commercial LLMs are acceptable. However, the business model presented needs to be sustainable for the Marine Corps. This means that commercial software package with a software license or subscription model is not ideal. Additionally, the Marine Corps needs to be able to iterate on the solution to be able to add new platforms, components, etc.
5/2/25  Q. You mentioned that the vehicle manual will be provided to the successful offerors as Government Furnished Information. Our team is wondering if the 3-D digital drafting / CAD files for the vehicle and/or parts can also provided? Or do you know if the vehicle manufacturer can provide them?
   A. The vehicles would be available to the successful offerors. The CAD models, etc. will not be available. They are owned by the OEM. However, the parts will be available for scanning.
4/29/25  Q. When discussing relevant past experiences, will the government accept ongoing projects or only fully completed projects?
   A. Identified experience can be prior work or ongoing. Both can show relevance to the topic.
4/25/25  Q. Is the ULTV manual available for applicants to use? If not, is there a publicly available technical manual for a reasonably equivalent vehicle that could be used in developing a solution without access to the actual ULTV?
   A. The manual is not available to the offerors for pre-submission of proposals. That information will be provided to the successful offerors as Government Furnished Information. The Polaris MRZR is a similar platform. If the manual is available, you may find some useful information in that document.
4/25/25  Q.
  1. Are there additional threshold or objective requirements regarding the communication between the maintainer and the expert during a reachback call? For example, does there need to be the ability to tag items in the maintainer’s view or send links to appropriate documentation?
  2. Are there current hardware or software options in use that we should consider as part of the analysis? For example, are some headsets already in use that function with the EMSS?
  3. Are only COTS SW products under consideration for the phase I analysis or is the performer expected to provide their own solution?
  4. For the Phase II, how do you expect that the prototype to be validated? Is this in-person user testing or is there a checklist of items that need to be addressed?
  5. What are the system requirements to demonstrate that the solution can be hosted on the EMSS? Are the requirements different to demonstrate that the solution can communicate from the host EMSS to another EMSS?
   A.
  1. At this time we do not see a need for the maintainer to tag an item so that the item is emphasized for the remote SME’s view. The item would be identified to the remote SME by the maintainer’s gaze and audible cue. Linking to appropriate documentation would be done by maintainer gaze or tag.
  2. We do not use any AR/VR/MR devices with EMSS. The selection of any device would the choice of the vendor as they determine what they see as being a suitable solution.
  3. There is no preference. Some level of development is anticipated. The use of COTS items can reduce some technical risk. We do not want to constrain innovation.
  4. The expectation is that there will be some combination of user evaluation and environmental testing.
  5. The EMSS laptop has a Windows 11 OS. The EMSS uses standard communications protocols.
4/25/25  Q.
  1. The solicitation states “capability can be expanded to maintenance actions on multiple Marine Corps platforms”. We are wondering if you can share what these Marine Corps platforms are? Is there specific hardware or software that is of interest?
  2. Is Unity compatible with Marine Corps platforms?
  3. Does the Navy have an image dataset that is already labeled? Or can the parts be easily ordered by us?
  4. Are there any issues with using an Apple Vision Pro?
  5. The solicitation mentions a Threshold and Objective, this means that detecting the four parts is the threshold and doing more than that is the objective correct?
  6. Can you provide any additional desirable maintenance actions to achieve objectives beyond the threshold?
  7. Can you provide any relevant specifications about the EMSS system that we need to integrate with our system?
   A.
  1. An example of additional platforms that we are interested in expanding to is the MTVR and JLTV. Additionally, we see potential usage of this technology in different areas across the Marine Corps, such as medical. There is not a specific set of hardware or software that we are interested in.
  2. Unity is compatible with the EMSS
  3. There are Technical Manuals and Schematics resident on the EMSS laptop for all supported platforms. These contain images and part numbers. EMSS has a connected version which has the ability to connect to the Marine Corps Enterprise Network. New parts can be ordered by accessing the Global Combat Support System (GCSS) website.
  4. No.
  5. There are multiple objective and threshold requirements. However, I believe the one you are referencing is this: Threshold – ability to replace ULTV brake pads, brake disk, fuel filter, and water separator; Objective – ability to replace the threshold and additional vehicle components. If you meet the threshold and provide the capability to add new components to the solution, then the objective is met.
  6. The ability to scan in new components and create new maintenance actions in the solution. We want the ability to be able to expand this capability.
  7. The EMSS contains a GETAC B360 Pro laptop which is running Windows 11 Enterprise. Specs: 12th Gen Intel Core i5-1250P 1.70 GHz, 16GB RAM, NVIDIA GeForce GTX 1650


[ Return ]