nrt

dev-nrt

NRT (Naplan Reporting Tool) is a command-line utility that processes results data from the NAPLAN online testing platform.

NRT takes one or more results-reporting-dataset (often called RRD) xml data files generated by the online platform, and processes them to produce a variety of csv reports that are easier for most users to work with.

There are some 60+ reports available from NRT, and the scope of reports that are run can be configured by the user, either by the use of command-line flags passed to the nrt application, or by editing the report configuration files that can be found in the /config folder of the distribution.

NRT is intended as a direct replacement for the existing nias2 naprrql application.

NRT aims to process results data faster than naprrql, and depending on configuration and input volume typically performs 6x to 50x faster.

basic usage

Download a pre-built binary distribution for your platform (Windows, Linux, Mac), from the releases area of this repository:

https://github.com/nsip/dev-nrt/releases/latest

Unpack the downloaded zip file, and you should see a folder structure like this:

/NRT-Mac-v0_0_1 
	nrt(.exe)
	├── config
	│   	└── internal
	└── testdata

in the folder is

• the nrt executable (nrt.exe if on windows),
• a subfolder called /config that contains the configuration files for each report produced by the tool
• a subfolder called /testdata that has two sample zipfiles of results data
  • n2sif.xml.zip; is the same sample file as accompanies nias2 naprrql
  • rrd.xml.zip; a much larger results file (100 schools, 5k students)

nrt expects results data files that it will process to be placed into an /in folder.

You can either create the folder yourself, or do the following:

Open a terminal window at the folder containing your nrt distribution.

Run nrt for the first time as

> ./nrt

for Linux/Mac or

> nrt.exe 

on Windows

this will produce the following message:

--- Ingesting results data:
2021/02/08 17:06:22 No results data *.zip *.xml.zip or *.xml files found in input folder in

but the folder layout of your installation will now look like this:

/NRT-Mac-v0_0_1 
nrt(.exe)
├── config
│   └── internal
├── in
├── kv
└── testdata

the /in folder has been created as has a /kv folder the app will use internally.

Copy either of the xml.zip files from the testdata area to the /in folder in order to have them processed by nrt.

(nrt will process multiple files so you can copy both if you wish to, it just means processing will take longer).

nrt does read zipped xml files, or you can unzip them (if you do so remember to delete the zipped version or nrt will process both). However, nrt is not able to open password-protected zip files.

now if you invoke nrt again, it will ingest and process the data file/s.

typical output as follows:

| => ./nrt
--- Ingesting results data:

Processing XML File:	(in/n2sif.xml.zip)
   0s [===================================================================>] 100%

	5190 data-objects parsed

2021/02/08 17:11:56 Results data ingest took 1.038s
--- Data stats:

	NAPStudentResponseSet: 1061
	NAPTestScoreSummary: 154
	NAPTestItem: 2310
	NAPCodeFrame: 22
	SchoolInfo: 10
	StudentPersonal: 250
	NAPTestlet: 174
	NAPTest: 22
	NAPEventStudentLink: 1187



--- Running Reports (-QA reports):

 Simple reports:                       0s [====================================================================] 100%
 Codeframe reports:                    0s [====================================================================] 100%
 Event-based reports:                  1s [====================================================================] 100%
 Student-based reports:                1s [====================================================================] 100%

2021/02/08 17:11:59 Report processing took 2.642s
______________________

First nrt ingests the data, and presents a set of summary statistics, then it proceeds to run the core report set, which are the most commonly used.

The indicator '(-QA reports)' shows that qa reports are not included in this set, this changes to (+QA reports) if qa reports have been requested by the user.

All generated reports are output to the /out folder which nrt creates. The folder structure after a full run of data ingest, and all reports would look like this:

| => 
/NRT-Mac-v0_0_1 
nrt(.exe)
├── config
│   └── internal
├── in
├── kv
├── out
│   ├── item_printing
│   ├── qa
│   ├── system_reports
│   └── writing_extract
└── testdata

A description of the various command-line options to configure the reporting output of nrt is available at any time by invoking ./nrt --help:

| => ./nrt --help

Usage of ./nrt:
  -allReports
    	runs every report including qa reports
  -coreReports
    	run core naplan reports (on by default) - (alias for n2 -reports flag)
  -forceIngest
    	always ingest data from files in the input folder (default true)
  -ingest
    	halt processing after data has been ingested
  -inputFolder string
    	folder containing results data files for processing (default "./in/")
  -itemLevel
    	include item-level reports - very detailed (alias for n2 -itemprint flag)
  -itemprint
    	include item-level reports - very detailed (alias for nrt -itemLevel flag)
  -qa
    	include QA reports in results processing
  -report
    	run core naplan reports - (alias for nrt -coreReports flag)
  -showProgress
    	show progress bars in console for report processing (default true)
  -skipIngest
    	don't ingest data again, move directly to processing of reports
  -writingExtract
    	(alias) run writing extract reports; generates writing inputs for marking systems
  -writingextract
    	run writing extract reports; generates writing inputs for marking systems

The -itemLevel/-itemprint flags (aliases of each other, for compatibility with naprrql command-line) are of particular note.

These flags invoke the most performance-heavy reports in the collection, they are output to the out/item_printing folder.

These reports are at the highest level of granularity, producing very large numbers of rows (event-based) and columns (student-based). Event-based generates a row for every item seen by every student in every testlet in every test. Student-based creates a row with columns per student containing personal information, event information, every and every test, testlet and item seen by the student, typically around 1400 columns for each student.

These reports were created through user demand, but if not explicitly needed, performance is far better without them.

Command-line flags can be used together in whatever combination helps most.

The -writingExtract/-writingextract flags if used on their own will result in only the writing extract reports being run. These are typically needed before the full rrd dataset is produced as they export writing results for ingest into external marking systems.