org.encog.neural.networks.training.propagation.sgd

Class BatchDataSet

java.lang.Object

org.encog.neural.networks.training.propagation.sgd.BatchDataSet

All Implemented Interfaces:

java.lang.Iterable<MLDataPair>, MLDataSet

public class BatchDataSet
extends java.lang.Object
implements MLDataSet

The BatchDataSet wraps a larger dataset and breaks it up into a series of batches. This dataset was specifically created to be used with the StochasticGradientDescent trainer; however, it should work with the others as well. It is important that the BatchDataSet's advance method be called at the end of each iteration, so that the next batch can be prepared. All Encog-provided trainers will detect the BatchDataSet and make this call. This dataset can be used in two ways, depending on the setting of the randomSamples property. If this value is false (the default), then the first batch starts at the beginning of the dataset, and following batches will start at the end of the previous batch. This method ensures that every data item is used If randomSamples is true, then each batch will be sampled from the underlying dataset (without replacement).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	BatchDataSet.BatchedMLIterator An iterator to be used with the BasicMLDataSet.

Constructor Summary

Constructors

Constructor and Description
BatchDataSet(MLDataSet theDataset, GenerateRandom theRandom) Construct the batch dataset.

Method Summary

Modifier and Type	Method and Description
void	add(MLData data1) This operation is not supported by this object.
void	add(MLData inputData, MLData idealData) This operation is not supported by this object.
void	add(MLDataPair inputData) This operation is not supported by this object.
void	advance() Advance to the next batch.
void	close() Close this datasource and release any resources obtained by it, including any iterators created.
MLDataPair	get(int index)
int	getBatchSize()
int	getCurrentIndex()
int	getIdealSize()
int	getInputSize()
void	getRecord(long index, MLDataPair pair) Read an individual record, specified by index, in random order.
long	getRecordCount() Determine the total number of records in the set.
boolean	isRandomBatches()
boolean	isSupervised()
java.util.Iterator<MLDataPair>	iterator()
MLDataSet	openAdditional() This will open an additional batched dataset.
void	setBatchSize(int theSize)
void	setCurrentIndex(int currentIndex) Set the current index, within a batch.
void	setRandomBatches(boolean randomBatches) Set if random batches should be generated.
int	size()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Detail

BatchDataSet

public BatchDataSet(MLDataSet theDataset,
                    GenerateRandom theRandom)

Construct the batch dataset.

Parameters:

theDataset - The source dataset.

theRandom - The random number generator.

Method Detail

setBatchSize

public void setBatchSize(int theSize)

Parameters:

theSize - Set the batch size, but not larger than the dataset.

getBatchSize

public int getBatchSize()

iterator

public java.util.Iterator<MLDataPair> iterator()

Specified by:

iterator in interface java.lang.Iterable<MLDataPair>

getIdealSize

public int getIdealSize()

Specified by:

getIdealSize in interface MLDataSet

Returns:

The size of the ideal data.

getInputSize

public int getInputSize()

Specified by:

getInputSize in interface MLDataSet

Returns:

The size of the input data.

isSupervised

public boolean isSupervised()

Specified by:

isSupervised in interface MLDataSet

Returns:

True if this is a supervised training set.

getRecordCount

public long getRecordCount()

Determine the total number of records in the set.

Specified by:

getRecordCount in interface MLDataSet

Returns:

The total number of records in the set.

getRecord

public void getRecord(long index,
                      MLDataPair pair)

Read an individual record, specified by index, in random order.

Specified by:

getRecord in interface MLDataSet

Parameters:

index - The index to read.

pair - The pair that the record will be copied into.

openAdditional

public MLDataSet openAdditional()

This will open an additional batched dataset. However, please note, the additional datasets will use a mersenne twister generator that is seeded by a long sampled from this object's random number generator.

Specified by:

openAdditional in interface MLDataSet

Returns:

An additional dataset.

add

public void add(MLData data1)

This operation is not supported by this object.

Specified by:

add in interface MLDataSet

Parameters:

data1 - NA

add

public void add(MLData inputData,
                MLData idealData)

This operation is not supported by this object.

Specified by:

add in interface MLDataSet

Parameters:

inputData - NA

idealData - NA

add

public void add(MLDataPair inputData)

This operation is not supported by this object.

Specified by:

add in interface MLDataSet

Parameters:

inputData - NA

close

public void close()

Close this datasource and release any resources obtained by it, including any iterators created.

Specified by:

close in interface MLDataSet

size

public int size()

Specified by:

size in interface MLDataSet

get

public MLDataPair get(int index)

Specified by:

get in interface MLDataSet

advance

public void advance()

Advance to the next batch. Should be called at the end of each training iteration.

getCurrentIndex

public int getCurrentIndex()

Returns:

The current index, within a batch.

setCurrentIndex

public void setCurrentIndex(int currentIndex)

Set the current index, within a batch.

Parameters:

currentIndex - The current index, within a batch.

isRandomBatches

public boolean isRandomBatches()

Returns:

True, if random batches are being used.

setRandomBatches

public void setRandomBatches(boolean randomBatches)

Set if random batches should be generated.

Parameters:

randomBatches - True, if random batches should be used.