edu.uci.jforests.eval.ranking

Class URiskAwareEval

java.lang.Object

edu.uci.jforests.eval.EvaluationMetric

edu.uci.jforests.eval.ranking.RankingEvaluationMetric

edu.uci.jforests.eval.ranking.URiskAwareEval

Direct Known Subclasses:

TRiskAwareSAROEval

public class URiskAwareEval
extends RankingEvaluationMetric

Implements the URisk risk-sensitive evaluation metric, as defined by Wang et al. See Lidan Wang, Paul Bennet and Kevyn Collin-Thompson, SIGIR 2012.

Implementation Details

It is assumed that the natural ordering of documents for each query defines the baseline.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	URiskAwareEval.TestURiskSwaps

Nested classes/interfaces inherited from class edu.uci.jforests.eval.ranking.RankingEvaluationMetric

RankingEvaluationMetric.SwapScorer

Constructor Summary

Constructors

Constructor and Description
URiskAwareEval(EvaluationMetric _parent, double alpha)

Method Summary

Modifier and Type	Method and Description
EvaluationMetric	getParentMetric()
RankingEvaluationMetric.SwapScorer	getSwapScorer(double[] targets, int[] boundaries, int trunc, int[][] labelCounts)
double	measure(double[] predictions, Sample sample)
double[]	measureByQuery(double[] predictions, Sample sample)

Methods inherited from class edu.uci.jforests.eval.ranking.RankingEvaluationMetric

computeNaturalOrderScores

Methods inherited from class edu.uci.jforests.eval.EvaluationMetric

isFirstBetter, largerIsBetter

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

URiskAwareEval

public URiskAwareEval(EvaluationMetric _parent,
                      double alpha)

Method Detail

measure

public double measure(double[] predictions,
                      Sample sample)
               throws java.lang.Exception

Overrides:

measure in class RankingEvaluationMetric

Throws:

java.lang.Exception

getSwapScorer

public RankingEvaluationMetric.SwapScorer getSwapScorer(double[] targets,
                                                        int[] boundaries,
                                                        int trunc,
                                                        int[][] labelCounts)
                                                 throws java.lang.Exception

Specified by:

getSwapScorer in class RankingEvaluationMetric

Parameters:

targets - Targets of all documents in dataset. Equals Dataset.*.length

boundaries - Start and end indices of each query within the dataset.

trunc - Truncation level for evaluation measure

labelCounts - How many of each label value

Throws:

java.lang.Exception - if things go pear shaped

measureByQuery

public double[] measureByQuery(double[] predictions,
                               Sample sample)
                        throws java.lang.Exception

Specified by:

measureByQuery in class RankingEvaluationMetric

Throws:

java.lang.Exception

getParentMetric

public EvaluationMetric getParentMetric()

Overrides:

getParentMetric in class RankingEvaluationMetric