edu.uci.jforests.eval.ranking

Class TRiskAwareFAROEval

java.lang.Object

edu.uci.jforests.eval.EvaluationMetric

edu.uci.jforests.eval.ranking.RankingEvaluationMetric

edu.uci.jforests.eval.ranking.URiskAwareEval

edu.uci.jforests.eval.ranking.TRiskAwareSAROEval

edu.uci.jforests.eval.ranking.TRiskAwareFAROEval

public class TRiskAwareFAROEval
extends TRiskAwareSAROEval

Implements the TRisk risk-sensitive optimisation metric called FARO, as defined by Dincer et al. See Taner Dincer, Craig Macdonald and Iadh Ounis, Hypothesis Testing for Risk-Sensitive Evaluation of Retrieval Systems, SIGIR 2014.

Implementation Details

It is assumed that the natural ordering of documents for each query defines the baseline.

Nested Class Summary

Nested classes/interfaces inherited from class edu.uci.jforests.eval.ranking.URiskAwareEval

URiskAwareEval.TestURiskSwaps

Nested classes/interfaces inherited from class edu.uci.jforests.eval.ranking.RankingEvaluationMetric

RankingEvaluationMetric.SwapScorer

Constructor Summary

Constructors

Constructor and Description
TRiskAwareFAROEval(EvaluationMetric _parent, double alpha)

Method Summary

Modifier and Type	Method and Description
RankingEvaluationMetric.SwapScorer	getSwapScorer(double[] targets, int[] boundaries, int trunc, int[][] labelCounts)

Methods inherited from class edu.uci.jforests.eval.ranking.TRiskAwareSAROEval

getEstimates, measure, T_measure

Methods inherited from class edu.uci.jforests.eval.ranking.URiskAwareEval

getParentMetric, measureByQuery

Methods inherited from class edu.uci.jforests.eval.ranking.RankingEvaluationMetric

computeNaturalOrderScores

Methods inherited from class edu.uci.jforests.eval.EvaluationMetric

isFirstBetter, largerIsBetter

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TRiskAwareFAROEval

public TRiskAwareFAROEval(EvaluationMetric _parent,
                          double alpha)

Method Detail

getSwapScorer

public RankingEvaluationMetric.SwapScorer getSwapScorer(double[] targets,
                                                        int[] boundaries,
                                                        int trunc,
                                                        int[][] labelCounts)
                                                 throws java.lang.Exception

Overrides:

getSwapScorer in class TRiskAwareSAROEval

Parameters:

targets - Targets of all documents in dataset. Equals Dataset.*.length

boundaries - Start and end indices of each query within the dataset.

trunc - Truncation level for evaluation measure

labelCounts - How many of each label value

Throws:

java.lang.Exception - if things go pear shaped