Agentic AI Script Writer for VR Simulations

N8N Agentic Workflow

Agentic AI workflow built in N8N, utilizing Google Gemini 2.5 Flash — Users initiate the flow via Slack chatbot and submit a design outline. The composer then utilizes the appropriate tool for each step based on the step. The user collaborates with AI to revise each step as needed via Slack, and the script segment is sent to a Google Doc once approved.

Outline for Lighting Technician Simulation as Input for Agentic Tool

Designer-generated outline which acts as the input for the agentic tool — Instructional designers are able to incorporate specific process instructions and design elements, which are then expanded into the full script. The specific tool for each step is indicated on the outline.

1. $IntroWriter
Introduction
-Lighting technician is at an indoor film set and mentions that the resto f the crew is on break after setting everything up

-all of the lights are off except for one lamp on a table next to a lighting console

-the stage is a barn scene in front of a green screen

-the coach alludes to needing to hurry to confirm everything is correct because the main 'talent' is going to arrive on scene shortly, coach refers to the 'talent' multiple times throughout the script

1.5 $PPEWriter

Interactive, no mannequin

User puts on hard hat

User needs work gloves, hardhat, and glasses

2. $StepWriter

User turns on main light and the lights on the film set click on in order making a dramatic sound, revealing the environment from the darkness

3. $StepWriter

User picks up tablet to review the lighting director's plan

4. $StepWriter

User flips switch to test the side lights

-the lightbulb in one of the side lights shatters and smoke is emitted from the light

5. $StepWriter

Coach says we can get this swapped quickly, and rolls a Step Steel Rolling Ladder under the light

6. $StepWriter

Teleport

User teleports to the top of the ladder

7. $StepWriter

User reaches up to fix the light, but cannot reach it

-coach says they will need to go up to the catwalk to fix the light

8. $StepWriter

Teleport

User teleports to the bottom of the catwalk ladder

9. $StepWriter

User climbs the ladder

10. $StepWriter

User goes to touch the light and upon contact, it falls but is caught by the safety chain

-coach mentions that that was a close call because the expensive film camera was right under the light

-coach tells user that the light fell because it was placed improperly on the truss with a yoke adapter

11. $StepWriter

Coach sets the light on a box next to the user

12. $StepWriter

User unscrews yoke adapter and coach takes it out of their hand

13. $StepWriter

User takes a trigger clamp and places it on the light

14. $StepWriter

Coach places the wing nut on and user tightens it by hand

-Coach mentions that he will replace the light bulb

15. $StepWriter

User brings the light to the correct spot on the truss and the coach tightens the wing nut on top

-coach mentions that the safety chain is also in place

-as soon as the light is up, people start to show up on the set and someone walks a cow to the middle of the scene

16. $StepWriter

Coach asks the user "Didn't I mention that the star of this movie was a cow?" then prompts the user to teleport to the set to confirm the incident lighting with a light meter

17. $StepWriter

Teleport

user teleports to the main set

18. $StepWriter

User picks up light meter, points it at the cow, and presses the trigger button to capture the reading

19. $StepWriter

The lighting director confirms that the light is correct and filming can get started

20. $OutroWriter

Slideshow and star rating

Segment of Prompting From Core Step Writing Tool

Segment of the Step Writer tool prompting — The segment demonstrates the few-shot prompting technique to teach the tool the internal conventions of the virtual reality design documentation. The examples were pulled from top-rated simulations in the career exploration product line. The types of voiceover lines were differentiated to allow the tool to know which lines are involved in a VR interaction and thus require an idle line, which is played if a user hesitates to perform an action. The prompting was developed with the least-to-most prompting approach to keep token usage and latency low.

Follow these instructions in order:

1 - For the step in the outline that specified the #StepWriter, identify whether the step states it is a teleport

2 - Write an active-tense step title which shows the Step # from the provided outline and a 1 to 5 word description of the step

3 - If the step is a teleport, write the script segment based on the teleport examples

4 - If the step is not a teleport, follow these instructions while writing the step:

- For any dialogue line giving background information, knowledge, or context, these start with [Context Coach VO:]

- For every dialogue line where the coach asks the user to do an action, these start with [Action Coach VO:]

- For all Action VO lines, there should be an alternative VO for if the user hesitates and does not perform the action; this VO should be more specific, written under each VO line that tells the user to do an action, and start with [Idle Coach VO:]

- If the Action Coach VO has two separate actions, create two Idle Coach VO lines (one for each action)

- If a tool is needed, it should say appear <tool>. If a user is to perform an action, it should say user <performs action>

5 - Revise the step as needed based on my requests. ONLY change the lines that I ask for revisions on, keep everything else identical

# Example output for non-teleport step

Step 6: Fasten Blocking with Nailer

[Context Coach VO:] We need to secure the truss with some nails.

[Action Coach VO:] Pick up the framing nailer.

[Idle Coach VO:] Pick up the framing nailer on the table.

user picks up framing nailer

[Action Coach VO:] Position it here.

[Idle Coach VO:] Position the framing nailer where I’m showing you.

user places nailer on position 1

[Action Coach VO:] And pull the trigger.

[Idle Coach VO:] Press the trigger to use the framing nailer.

user presses trigger button

SFX: Pneumatic nailgun firing into wood

If user drops framing nailer:

[Conditional Coach VO:] Oops! Pick up the framing nailer.

[Context Coach VO:] Good! We just need one more on this side.

***

# Example output for non-teleport step

Step 9: Measure the Pipe

[Context Coach VO:] We’ll cut the piece we need from this twenty-one foot pipe. I already made sure it’s aligned correctly.

[Context Coach VO:] Ready to mark it up?

appear tape measure

[Action Coach VO:] Grab the tape measure.

[Idle Coach VO:] Pick up the tape measure from the table.

user picks up tape measure

appear ghost tape measure on end point

[Action Coach VO:] Position the hook on the edge of the pipe.

[Idle Coach VO:] Position the hook on the edge of the pipe, where I’m showing you.

user places tape measure on end

[Action Coach VO:] Drag the tape measure along the pipe until you’ve reached eleven and three-quarters inches.

[Idle Coach VO:] Drag the tape measure along the pipe until you’ve reached eleven and three-quarters inches.

user drags tape measure to end point

[Context Coach VO:] I’ll hold the tape measure in place so you can mark the pipe.

***

# Example output for non-teleport step

Step 3: Inject Medication Into IV Bag

[Context Coach VO:] I'll replace the filter needle with an I-V hypodermic needle.

*animation of filter needle being replaced by IV needle

[Context Coach VO:] To complete the infusion, you'll inject the medication into the left port on the I-V bag.

[Action Coach VO:] Pick up the syringe and insert it into the I-V bag port.

[Idle Coach VO:] Pick up the syringe.

[Idle Coach VO:] Insert the needle into the I-V bag port.

user picks up syringe

user inserts syringe into IV port

[Action Coach VO:] Grab the plunger on the syringe and push slowly to inject the medication into the I-V bag.

[Idle Coach VO:] Grab the plunger on the syringe where I’m showing you, and push slowly to inject the medication.

*user slides plunger

[Context Coach VO:] Good job completing the I-V infusion! Now we need to bring it to the pharmacist to review.

[Context Coach VO:] I’ll clean up and meet you in there.

***

# Example output for teleport

Step 7: Teleport to Platform

appear teleport button UI

[Action Coach VO:] Push the button to teleport to the boom lift.

[Idle Coach VO:] Push the button on the teleport panel.

User presses the teleport button

User appears in Playspace 2

Playspace 2: Standing on aerial work platform (boom lift)

[Context Coach VO:] You’re safely tied off and ready to operate the lift.

***

# Example output for teleport

Step 5: Teleport to Hill

[Context Coach VO:] Let’s climb down and take a look at the tower from further away.

appear teleport button UI

[Action Coach VO:] Push the teleport button.

[Idle Coach VO:] Push the button on the teleport panel.

User presses the teleport button

User appears in Playspace 2

Playspace 2: Standing on a hillside overlooking the tower

[Context Coach VO:] It usually takes a month to build a tower but because of the emergency situation, we got it back up in just two weeks, complete with antennas and cabling.

***

Segment From Script Output

Introduction and first step of the generated Lighting Technician VR script — The script segment was produced following collaborative refinement with the user via a chat interface in Slack. The segment demonstrates the appropriate tagging of VO lines, the creation of idle lines for all action lines, adheres to the design document standards, and captures the authentic tone of the coach that is expected for our products.

Final Lighting Technician Virtual Reality Simulation

Playthrough of the virtual reality Lighting Technician simulation on a Meta Quest 2 headset — The final simulation was fully developed by a vendor using the automatically generated script. This expedited production time, decreased production cost, and ensured the tone, story arc, and quality standards of the company were honored.