AnnotationStandard - Revision history

JuliaLongtin: /* Phase 1: Recording */

2026-01-25T23:20:54Z

Phase 1: Recording

← Older revision		Revision as of 23:20, 25 January 2026
Line 101:		Line 101:
	=== Phase 1: Recording ===		=== Phase 1: Recording ===

	'''Objective:''' Create a terminal recording of meaningful software ~~deployment activity~~.		'''Objective:''' Create a terminal recording of meaningful operations work, software development, or other terminal baased productive activities.
	'''Who:''' Person A (one or more individuals)		'''Who:''' Person A (one or more individuals)

	'''Steps:'''		'''Steps:'''
	# Set up the recording environment:		# Set up the recording environment:
	#* Use a Debian Stable VM (see Appendix D)		#* Use a Debian Stable system (see Appendix D)
	#* Upgrade to unstable if required packages are missing		#* Upgrade to unstable if required packages are missing
	# VMs may be reused from previous deployments, or may be new		# VMs may be reused from previous deployments, or may be new
	# Start the <code>asciinema</code> recording client		# Start the <code>asciinema</code> recording client
	# Attempt to ~~deploy a piece of GNU software~~		# Attempt to do useful work
	# Keep all input/output inside the terminal:		# Keep all input/output inside the terminal:
	#* Use CLI web tools (<code>curl</code>, <code>wget</code>, <code>lynx</code>, <code>links</code>, etc.)		#* Use CLI web tools (<code>curl</code>, <code>wget</code>, <code>lynx</code>, <code>links</code>, etc.)
	# Stop recording when ~~deployment completes~~		# Stop recording when work is done

	'''Output:''' <code>.cast</code> asciinema recording file		'''Output:''' <code>.cast</code> asciinema recording file
Line 120:		Line 120:
	* Using GUI tools		* Using GUI tools
	* Stopping recording too early		* Stopping recording too early
			* Forgetting --stdin on asciinema's command line
	----		----

JuliaLongtin at 23:18, 25 January 2026

2026-01-25T23:18:33Z

← Older revision		Revision as of 23:18, 25 January 2026
Line 1:		Line 1:
	= ~~Librecode~~ Annotation Standard =		= Annotation Standard =

	'''Document Version:''' 2.0		'''Document Version:''' 2.0

JuliaLongtin at 23:18, 25 January 2026

2026-01-25T23:18:02Z

← Older revision		Revision as of 23:18, 25 January 2026
Line 62:		Line 62:
	* Provide an annotation standard.		* Provide an annotation standard.
	* Collect guidance for people participating in annotation.		* Collect guidance for people participating in annotation.

	~~An '''annotated session''' is defined as an asciinema recording, with annotations added to it, conforming to a version of this document.~~

	=== What are Annotated Sessions? ===		=== What are Annotated Sessions? ===
Line 76:		Line 74:
	=== Why Create Them? ===		=== Why Create Them? ===

	The ~~Librecode~~ project aims to train Large Language Models (LLMs) to understand and work with terminal recordings. Annotated sessions serve as training data, teaching models to:		The AutoDocs project aims to train Large Language Models (LLMs) to understand and work with terminal recordings. Annotated sessions serve as training data, teaching models to:
	* Recognize common deployment patterns		* Recognize common deployment patterns
	* Understand error recovery strategies		* Understand error recovery strategies

JuliaLongtin at 21:02, 25 January 2026

2026-01-25T21:02:12Z

← Older revision		Revision as of 21:02, 25 January 2026
Line 1:		Line 1:
	~~```mediawiki~~
	= Librecode Annotation Standard =		= Librecode Annotation Standard =

Line 400:		Line 399:
	=== Appendix D: Debian Stable VM details ===		=== Appendix D: Debian Stable VM details ===
	(See project-specific VM setup instructions for the current cohort/repository; keep this appendix synchronized with those instructions.)		(See project-specific VM setup instructions for the current cohort/repository; keep this appendix synchronized with those instructions.)
	~~```~~

ArthurWolf: Migrating missing data from the older cryptpad.fr annotation standard.

2026-01-22T19:35:08Z

Migrating missing data from the older cryptpad.fr annotation standard.

Show changes

ArthurWolf: Initial port from the gist at https://gist.githubusercontent.com/arthurwolf/c4521a8baa306c016efd50ee23ffe56e/raw/26081cbc230641b004554778eefb11eee2a2fcf3/annotation.md , conversion from Markdown to MediaWiki formats

2026-01-11T23:39:31Z

Initial port from the gist at https://gist.githubusercontent.com/arthurwolf/c4521a8baa306c016efd50ee23ffe56e/raw/26081cbc230641b004554778eefb11eee2a2fcf3/annotation.md , conversion from Markdown to MediaWiki formats

New page

= Librecode Annotation Standard =

'''Document Version:''' 2.0
'''Last Updated:''' 2025-10-01
'''Target Audience:''' Session recorders, annotators, reviewers

----

== Table of Contents ==
# [[#Glossary|Glossary]]
# [[#Overview & Purpose|Overview & Purpose]]
# [[#The Annotation Process|The Annotation Process]]
# [[#Annotation Standard|Annotation Standard]]
# [[#Application Guidelines|Application Guidelines]]
# [[#Technical Specification|Technical Specification]]
# [[#Quick Reference|Quick Reference]]
# [[#Complete Worked Example|Complete Worked Example]]
# [[#Appendices|Appendices]]

----

== Glossary ==

'''Session'''
An asciinema recording of a terminal session, typically showing the deployment or configuration of software.

'''Annotated Session'''
A session that has structured annotations added to it, conforming to this specification.

'''Recording'''
The raw asciinema capture of terminal activity.

'''Annotation'''
Structured metadata added to a recording that describes what is happening, why, and the outcome.

'''Timeline'''
A hierarchical structure in the annotator tool where annotations are organized. Different goal levels exist on separate timelines.

'''Goal Hierarchy'''
The nested structure of goals, subgoals, and sub-subgoals that describe the overall task and its component steps.

'''Annotation Tag'''
A specific marker type (e.g., <code>goal</code>, <code>mistake</code>, <code>successResult</code>) used to categorize parts of the session.

'''Person A (Recorder)'''
The individual(s) who created the original terminal session recording.

'''Person B (Annotator)'''
The individual who interviews Person A and adds structured annotations to the recording.

'''.session File'''
A zip file containing the annotated recording, audio transcriptions, and metadata conforming to this specification.

----

== Overview & Purpose ==

=== What are Annotated Sessions? ===

An '''annotated session''' is an asciinema recording of terminal activity (typically software deployment) that has been enriched with structured annotations describing:
* What actions were performed
* Why those actions were taken
* Whether goals were achieved
* What mistakes occurred and their consequences
* What discoveries changed the approach

=== Why Create Them? ===

The Librecode project aims to train Large Language Models (LLMs) to understand and work with terminal recordings. Annotated sessions serve as training data, teaching models to:
* Recognize common deployment patterns
* Understand error recovery strategies
* Learn from human problem-solving approaches
* Eventually perform automatic annotation

Once sufficient manual annotations exist, trained models can automatically annotate new sessions, enabling advanced tooling like automated documentation generation.

=== Document Purpose ===

This document provides:
# An annotation process for creating annotated sessions
# An annotation standard defining what to annotate and how
# Guidelines for annotators to ensure consistency and quality

----

== The Annotation Process ==

The process of creating an annotated session involves four distinct phases:

<pre>
Phase 1: Recording → Phase 2: Interview → Phase 3: Annotation → Phase 4: Submission
</pre>

=== Phase 1: Recording ===

'''Objective:''' Create a terminal recording of meaningful software deployment activity.
'''Who:''' Person A (one or more individuals)

'''Steps:'''
# Set up the recording environment:
#* Use a Debian Stable VM (see Appendix D)
#* Upgrade to unstable if required packages are missing
# Start the <code>asciinema</code> recording client
# Attempt to deploy a piece of GNU software
# Keep all input/output inside the terminal:
#* Use CLI web tools (<code>curl</code>, <code>wget</code>, <code>lynx</code>, etc.)
# Stop recording when deployment completes

'''Output:''' <code>.cast</code> asciinema recording file

'''Common Issues:'''
* Using GUI tools
* Stopping recording too early

----

=== Phase 2: Interview ===

'''Objective:''' Ensure the annotator fully understands what was done and why.
'''Who:''' Person A and Person B

'''Steps:'''
# Record audio of both sides separately
# Review the recording together
# Explain what was done and why
# Record follow-up clarification conversations if needed

'''Output:'''
* Audio recordings
* Full understanding of actions and rationale

'''Why Separate Audio Recordings?'''
* Better transcription quality
* Clear speaker attribution
* Higher-quality training data

----

=== Phase 3: Annotation ===

'''Objective:''' Add structured annotations using the annotator tool.
'''Who:''' Person B

'''Steps:'''
# Load the recording
# Create timelines for goal hierarchy levels
# Apply annotations per standard
# Review completeness
# Note spec version used

'''Output:''' Annotated recording

----

=== Phase 4: Submission ===

'''Objective:''' Package and submit the session.

'''Steps:'''
# Transcribe audio with Whisper
# Create <code>.session</code> archive
# Include:
#* <code>session.yaml</code>
#* <code>recording.asciinema</code>
#* Audio + transcriptions
# Upload to repository

---