DataPipe
, DataPipeForked
, DataSourceIndependent
, IDataSink
, IDataSource
AccumulatorAverageCollapsingLogAB
public class AccumulatorAverageCollapsingLog extends DataAccumulator implements DataSourceIndependent
This class also uses a method to improve the block statistics for large. All block averages are kept for a particular block size (n1 blocks of size b1). Statistics for that block are computed from that data. Block averages for blocks of the next larger size (2*n1) are computed by randomly selecting n1/2 pairs of block values from the b1 set (without including any block value more than once). After using all block values, n1/2 additional pairs are taken from the full b1 set, such that n1 blocks of size b1*2 are obtained, instead of n1/2 blocks that would normally exist. Statistics for block size b1*2 are computed from these n1 blocks. This process is repeated for each block size such that the largest block size (which would normally have 1 block) has n1 blocks.
This accumulator can only operate on Data with a single value.
Modifier and Type | Field | Description |
---|---|---|
protected DataFunction |
averages |
|
protected double[] |
blockSums |
|
protected long |
count |
|
protected int |
initialSeed |
|
protected java.util.ArrayList<java.lang.Integer> |
intArrayList |
|
protected AccumulatorAverageCollapsing |
lAvg |
|
protected DataDouble |
lData |
|
protected DataDouble.DataInfoDouble |
lDataInfo |
|
protected double[][] |
lSums |
|
protected DataDoubleArray |
nData |
|
protected DataDoubleArray.DataInfoDoubleArray |
nDataInfo |
|
protected int |
nMoments |
|
protected int |
nRawData |
|
protected int |
nRawDataDoubles |
|
protected DataTag |
nTag |
|
protected IRandom |
random |
|
protected double[] |
rawData |
|
protected double[] |
rawData2 |
|
protected double[] |
rawData3 |
|
protected int |
rawDataBlockSize |
|
protected DataFunction |
skew |
|
protected DataFunction |
stdev |
|
protected double[][] |
sums |
|
protected boolean |
withReplacement |
active, hasUnpushedData, pushInterval, putCount
dataInfo, dataSink, tag
Constructor | Description |
---|---|
AccumulatorAverageCollapsingLog() |
|
AccumulatorAverageCollapsingLog(int nMoments) |
With this constructor, the accumulator will use an internal RNG with a
set seed such that each resampling of the data will be identical (the
same set of samples will go into the averages and the standard
deviations, etc).
|
AccumulatorAverageCollapsingLog(IRandom random) |
|
AccumulatorAverageCollapsingLog(IRandom random,
int nMoments) |
Modifier and Type | Method | Description |
---|---|---|
boolean |
addData(IData data) |
Add the given values to the sums and block sums.
|
protected void |
doSums(double[] subSums,
double v) |
Performs the block sum after
|
void |
enableRawLogStatistics() |
|
protected void |
ensureHappyIntArrayList() |
Refills inArrayList if it's empty.
|
IData |
getAverageLogs() |
Returns the average natural logs of the block averages, for all block
sizes.
|
IData |
getAverages() |
Returns the averages for all block sizes
|
long |
getCount() |
|
IData |
getData() |
|
int |
getIndependentArrayDimension() |
Returns the number of independent data dimensions
|
DataDoubleArray |
getIndependentData(int i) |
Returns the X data for the given dimension
|
DataDoubleArray.DataInfoDoubleArray |
getIndependentDataInfo(int i) |
Returns the DataInfo for the given dimension
|
DataTag |
getIndependentTag() |
Returns the tag associated with this DataSource.
|
int |
getNumRawData() |
|
double[] |
getRawData() |
|
int |
getRawDataBlockSize() |
|
AccumulatorAverage |
getRawLogAccumulator() |
|
double |
getRawLogDataCorrelation() |
Returns the correlation between consecutive values of the log of the "raw"
data held by the accumulator.
|
double |
getSampleLogVariance() |
|
double |
getSampleVariance() |
|
IData |
getSkewLog() |
Returns the standard deviation of the log of the block averages, for all
block sizes.
|
IData |
getStdev() |
Returns the standard deviation of the block averages, for all block
sizes.
|
IData |
getStdevLog() |
Returns the standard deviation of the log of the block averages, for all
block sizes.
|
static void |
main(java.lang.String[] args) |
|
IData |
processData(IData inputData) |
Returns getData() every pushInterval times it is invoked, otherwise returns null
(thereby preventing transmission of data to the next DataSinks).
|
IDataInfo |
processDataInfo(IDataInfo incomingDataInfo) |
Informs this DataProcessor of the DataInfo for the Data it will be
processing.
|
protected void |
reblockData() |
Reblock data from rawData2 into rawData3, then copy it back to rawData2
|
protected void |
reblockData(int n) |
Reblock data from rawData into rawData2, taking blocks of size n
|
void |
reset() |
Resets all sums to zero.
|
protected void |
resizeData(int newSize) |
|
void |
setNumRawDataDoubles(int newNumRawDataDoubles) |
Sets the number of times the random block generation method is used.
|
getPushInterval, isActive, setActive, setPushInterval, toString
getDataInfo, getTag, putData, putDataInfo
addDataSink, getDataSink, getDataSinks, removeDataSink, setDataSink, setDataSinks
putData, putDataInfo
getDataInfo, getTag
protected final DataTag nTag
protected final IRandom random
protected final int nMoments
protected final boolean withReplacement
protected long count
protected double[] blockSums
protected double[][] sums
protected double[][] lSums
protected DataFunction averages
protected DataFunction stdev
protected DataFunction skew
protected DataDoubleArray.DataInfoDoubleArray nDataInfo
protected DataDoubleArray nData
protected double[] rawData
protected double[] rawData2
protected double[] rawData3
protected int nRawData
protected int rawDataBlockSize
protected int nRawDataDoubles
protected java.util.ArrayList<java.lang.Integer> intArrayList
protected int initialSeed
protected AccumulatorAverageCollapsing lAvg
protected DataDouble lData
protected DataDouble.DataInfoDouble lDataInfo
public AccumulatorAverageCollapsingLog()
public AccumulatorAverageCollapsingLog(int nMoments)
nMoments
- the number of moments handled by the accumulator.
should be 2 (standard deviation) or 3 (stdev + skew).public AccumulatorAverageCollapsingLog(IRandom random)
public AccumulatorAverageCollapsingLog(IRandom random, int nMoments)
nMoments
- the number of moments handled by the accumulator.
should be 2 (standard deviation) or 3 (stdev + skew).public static void main(java.lang.String[] args)
public void setNumRawDataDoubles(int newNumRawDataDoubles)
public IData processData(IData inputData)
DataAccumulator
processData
in class DataAccumulator
inputData
- data to be processed.public boolean addData(IData data)
addData
in class DataAccumulator
data
- the data to be processedpublic long getCount()
protected void resizeData(int newSize)
public IData getData()
getData
in interface IDataSource
getData
in class DataAccumulator
public IData getAverages()
public IData getAverageLogs()
protected void reblockData()
protected void reblockData(int n)
protected void ensureHappyIntArrayList()
public IData getStdev()
public IData getStdevLog()
public IData getSkewLog()
protected void doSums(double[] subSums, double v)
public int getNumRawData()
public double[] getRawData()
public double getRawLogDataCorrelation()
public double getSampleVariance()
public double getSampleLogVariance()
public int getRawDataBlockSize()
public void reset()
reset
in class DataAccumulator
public IDataInfo processDataInfo(IDataInfo incomingDataInfo)
DataProcessor
processDataInfo
in class DataProcessor
incomingDataInfo
- the DataInfo of the Data that will be input to this
DataProcessorpublic int getIndependentArrayDimension()
DataSourceIndependent
getIndependentArrayDimension
in interface DataSourceIndependent
public DataDoubleArray getIndependentData(int i)
DataSourceIndependent
getIndependentData
in interface DataSourceIndependent
public DataDoubleArray.DataInfoDoubleArray getIndependentDataInfo(int i)
DataSourceIndependent
getIndependentDataInfo
in interface DataSourceIndependent
public DataTag getIndependentTag()
DataSourceIndependent
getIndependentTag
in interface DataSourceIndependent
public void enableRawLogStatistics()
public AccumulatorAverage getRawLogAccumulator()