While making some parts of this website and some authoring other tools I needed to dabble a little with the OSRS cache. For example, to help build the OSRSBox Item Database project (osrsbox-db), I needed to extract all items from the OSRS cache. Along the way, I learned a lot about the cache structure, the type of information it contains and various tools that are available to help parse the information in the cache. This page documents some of the more useful information I found to hopefully help someone else who is interested.
This post starts with a brief discussion about the OSRS cache, then outlines how to use the open source RuneLite project to extract item definitions and npc definitions from the cache.
The OSRS Cache structure is pretty much the same as the very well know #317 Runescape 2 cache format. All cache files are stored in the home directory of the user. In Microsoft Windows (Vista onwards), the cache files are stored in the following directory:
Similarly, on Linux-based systems the cache files are stored in the following directory:
Inside the cache directory, there are a number of files, but there are only two primary file types/formats. These file types are easily identified by their corresponding file extension, namely:
.dat
extension.idx
extensionThe following sections discuss each of these file type/formats in further detail.
The primary data file is named main_file_cache.dat2
. This file contains the entire OSRS cache including information about items, npcs, images, sounds, models, and animations. However, the file is stored in a complex structure and you cannot just open the main_file_cache.dat2
file and view images or extract music. The file is made up of various blocks of data, some are compressed, some are not, and the information is not trivial to extract. To extract anything from the main_file_cache.dat2
file you must know the location of the data blocks. This information is provided by index files.
As the name suggests, the index file is an index for the main_file_cache.dat2
file. In OSRS there are currently 17 index files named consecutively from main_file_cache.idx0
to main_file_cache.idx16
, as well as one index file named main_file_cache.idx255
. The table below documents each index file, and the data that the index file points to.
Index file name | Content |
---|---|
main_file_cache.idx0 | Skeleton |
main_file_cache.idx1 | Skin |
main_file_cache.idx2 | Config (See below) |
main_file_cache.idx3 | Interface |
main_file_cache.idx4 | Sound Effects 1 |
main_file_cache.idx5 | Landscape/Maps |
main_file_cache.idx6 | Music 1 |
main_file_cache.idx7 | Models |
main_file_cache.idx8 | Sprites |
main_file_cache.idx9 | Texture |
main_file_cache.idx10 | Huffman |
main_file_cache.idx11 | Music 2 |
main_file_cache.idx12 | Client scripts |
main_file_cache.idx13 | Fonts |
main_file_cache.idx14 | Sound Effects 2 |
main_file_cache.idx15 | Sound Effects 3 |
main_file_cache.idx16 | Unknown |
main_file_cache.idx255 | Unknown |
The config file, or main_file_cache.idx2
contains highly useful information.
Index 2 block | Content |
---|---|
0 | Unknown |
1 | Underlay Definition |
2 | Empty |
3 | IdentityKit |
4 | Overlay Definition |
5 | Unknown |
6 | Object Definition |
7 | Empty |
8 | Mapped Values |
9 | NPC Definition |
10 | Item Definition |
11 | Empty |
12 | Animation Definition (sequence) |
13 | Animated Graphic Definition (Spot Animation/GFX) |
14 | VarpBit Definition |
15 | Empty |
16 | Varp Definition |
RuneLite is a suite of open source tools for Old School RuneScape. The project provides a free, open-source and super fast client for Old School RuneScape. In addition, the creators also provide a cache extraction tool and used to provide a client deobfuscator. This documentation focuses on the cache tool for extracting information from the OSRS cache, and dabbles a little later with the deobfuscator (using an old version from the RuneLite project) for extracting cache version information. The RuneLite homepage provides a summary of the project and precompiled downloads for the client. The RuneLite GitHub project page is the official repository to get the source code for the project.
The RuneLite client comes as a pre-compiled download (a .jar
file). However, the other tools that are part of the suite of tools do not come precompiled. Therefore, this section documents how to compile the tools. Although this is a pretty straight-forward process, it may be difficult for someone without much programming experience.
The following instructions were performed on Microsoft Windows 10 64-bit. However, the instructions should be suitable for other Microsoft Windows operating systems and also adaptable to other operating systems such as Linux. The following software is required: 1) Git; 2) Netbeans; 3) Java.
Now, clone the RuneLite repository using Git Bash. For those new to git, you need to open Git Bash using the Windows Explorer context menu. In Windows Explorer, browse to a folder where you want to down the RuneLite repository, right-click a white-space area in Windows Explorer, and select Git Bash. Then enter the following command into the Git Bash command prompt to download the source code from the RuneLite GitHub repository:
Excellent, we have everything ready to go.
Open the NetBeans IDE from the Windows Start Menu. We will start by making a couple of configurations to Netbeans to make life easier. Firstly, we will disable testing, mainly because it takes a very long time to compile the project with additional tests. Perform the following tasks:
Now, open the RuneLite repository in the NetBeans IDE:
Before we can build the actual cache tool, we need to modify the source code a little. Since RuneLite is a suite of tools, we need to configure the Maven project to only build the Cache tool. Using the navigation menu, open the following file:
You need to add a line in the pom.xml
file to specify the main class. With the main class, the tool cannot be run correctly. You need to add the following entry:
The entry needs to be in the <plugin>
section, specifically in the maven-assembly-plugin
section. Inside this XML entry is a configuration
tag, then a archive
tag. Basically, you need to add a manifest
tag that states that the resultant build file (.jar
), should use the Cache main class as the program entry point. Without this addition, you will get a no main manifest attribute
error when running the .jar
file. The following snippet displays a section of the pom.xml
file, and what it should look like after the entry is added.
Great! Now you can build the cache tool. Perform the following tasks:
This will compile the Cache tool and produce a .jar
file that can be executed. The build process may take a while for the first build because a collection of dependencies need to be downloaded. The .jar
file for the cache tool is available in the following directory:
Two different versions will be compiled:
cache-1.4.11-SNAPSHOT-jar-with-dependencies.jar
cache-1.4.11-SNAPSHOT.jar
As illustrated by the naming conventions, one has a .jar
build with all project dependencies. The other is a standalone .jar
file without dependencies. It is recommended to the use the cache-1.4.11-SNAPSHOT-jar-with-dependencies.jar
file to avoid any dependency issues.
The cache tool has support for extracting the following types of data from the OSRS cache:
.dat
filesYou can run the newly build cache tool by using the Windows Command environment. The following steps should help:
cmd
and left-click to openThe cache tool has the following general syntax:
Where each of the inputs are explained below:
<cache-location>
is the location of the OSRS cache. This is usually in the home directory; for example: C:\Users\<username>\jagexcache\oldschool\LIVE\
. In the following command examples, the username is: ph01l
.<cache-extraction-option>
is the type of information that you want to extract. Valid options are -items
, -npcs
, -objects
and -sprites
.<output-directory>
is the directory to save the output. Sometimes this will be created automatically by the tool, and sometimes you will have to manually create it. In the following command examples, as output is saved to the Desktop
folder for the user named ph01l
.Examples are helpful for this type of tool, where the command can be complex. The following five snippets provide examples of how to run the RuneLite cache tool.
You can also run the cache tool directly in Netbeans. However, to achieve this you must provide command line arguments within the NetBeans IDE. This is complex to set up, but simple after it is configured. To achieve this, in the bottom-right pane (the output/debugging pane) there is a little icon that looks like a fast-forward button - colored (yellow). There are two icons which look the same, it is the bottom one! When hovered over, it will read: Re-run with different parameters. This button allows us to enter command line arguments when running the program. The first line needs to have the arguments added to it. In the original format, the configuration has the following line:
If you want to Run the cache tool directly in Netbeans you need to add additional arguments. For example, to dump items you can use the following arguments:
The information I wanted to extract from the OSRS Cache was the item definitions. I needed this information to get an up-to-date and complete list of all items in OSRS. The code snippet below shows one item definition that has been extracted from the cache. The specific item has the ID number of 22111
and is the Dragonbone necklace
.
As you can see there is a variety of information - some useful, some not. The usefulness of the information really depends on what information you are after!