Skip to the content.

Quick Reference (Java)

Contents

Basic Template

import core.data.*
...
DataSource ds = DataSource.connect("<URL>");
ds.load();
...
ds.fetch...("<field>");  // see fetch...() methods below

Template for Processing

import core.data.*;
...
void setup() {
    DataSource.initializeProcessing(this);
    ...
} Then use `connect`, `load`, and `fetch...` as above.

Examining Available Data

After ds.load():

ds.printUsageString() 

To test if field paths are valid:

ds.hasFields("...", ...)

To get an array (or list) of available top-level field names:

String[] fields = ds.fieldNames();
// or
List<String> fields = ds.fieldNamesList();

Other Connection Methods

Specify a data format ("CSV", "XML", "JSON", etc.):

DataSource ds = DataSource.connectAs("<FORMAT>", "<URL>");

Connect using a data specification file (e.g. provided by instructor):

DataSource ds = DataSource.connectUsing("<URL>");

Connection Parameters

Some data sources may require additional parameters to construct the URL. Use setParam("<name>", "<value>") after the connect and before load. For example:

ds.setParam("format", "raw");

Data Source Options

Some data sources provide (or require) additional information to process them once they have been downloaded. The available options are format-specific and are listed by enabling verbose usage info:

ds.printUsageString(true);

Use

ds.setOption("<name>", "<value>");

For example (with a CSV data source):

ds.setOption("header", "ID,Name,Call sign,Country,Active");

Selecting From a .zip Archive

To use a file that is one of several in a ZIP archive, set the “file-entry” option:

ds.setOption("file-entry", "FACTDATA_MAR2016.TXT");

Cache Control

Control frequency of caching (or disable it):

ds.setCacheTimeout(<seconds>); 
// may use  CacheConstants.NEVER_CACHE
//      or  CacheConstants.NEVER_RELOAD (always caches)

Show where files are cached:

System.out.println(ds.getCacheDirectory());

Clear all cache files (for all data sources):

ds.clearENTIRECache();

Disable Download Progress Display

To turn off the dots that are printed while files are being downloaded:

DataSource.showDownloadProgress(boolean)

Note, this is a global setting and will apply to all data sources that are loaded after this statement has been executed.

View Preferences

Launch preferences GUI window.

DataSource.preferences();

When preferences are saved, the program will immediately terminate and exit. Comment out or delete the statement above to enable the program to continue running as usual.

Using an Iterator

DataSourceIterator iter = ds.iterator();
while (iter.hasData()) {
    String name = iter.fetchString("Name");
    boolean active = iter.fetchBoolean("Active");
    System.out.println(name + ": " + active);
    iter.loadNext();
}

Fetching Data

Extract data by field names/paths using the appropriate method(s):

// PRIMITIVE TYPE VALUES
public boolean fetchBoolean(String key);
public byte fetchByte(String key); 
public char fetchChar(String key);
public double fetchDouble(String key);
public float fetchFloat(String key);
public int fetchInt(String key);
public String fetchString(String key);
	
// ARRAYS
public boolean[] fetchBooleanArray(String key);
public byte[] fetchByteArray(String key);
public char[] fetchCharArray(String key);
public double[] fetchDoubleArray(String key);
public float[] fetchFloatArray(String key);
public int[] fetchIntArray(String key);
public String[] fetchStringArray(String key);
	
// LISTS
public ArrayList<Boolean> fetchBooleanList(String key);
public ArrayList<Byte> fetchByteList(String key);
public ArrayList<Character> fetchCharList(String key);
public ArrayList<Double> fetchDoubleList(String key);
public ArrayList<Float> fetchFloatList(String key);
public ArrayList<Integer> fetchIntList(String key);
public ArrayList<String> fetchStringList(String key);

// OBJECTS (of any class you name - the order of key names
//          should match a constructor of the class)
public <T> T fetch(String clsName, String... keys);
public <T> ArrayList<T> fetchList(String clsName, String... keys);
public <T> T[] fetchArray(String clsName, String... keys);