PLearn 0.1
Public Member Functions | Protected Attributes | Private Types
PLearn::SDBVMFieldICBCTargets Class Reference

#include <SDBVMat.h>

Inheritance diagram for PLearn::SDBVMFieldICBCTargets:
Inheritance graph
[legend]
Collaboration diagram for PLearn::SDBVMFieldICBCTargets:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 SDBVMFieldICBCTargets (Schema schema, bool use_roadstar, bool add_claims_sum_column, bool rescale_by_interval, bool rescale_by_start_date, Mat &start_date_rescaling_values, const string &targetname="ALL")
virtual void convertField (const SDBWithStats &sdb, const Row &row, const Vec &output) const
virtual int fieldWidth () const
 Width that the field occupies in the VMat.

Protected Attributes

bool use_roadstar_
bool add_claims_sum_column_
bool rescale_by_interval_
bool rescale_by_start_date_
Mat start_date_rescaling_values_
string targetname_
int reference_start_date_year_
int reference_start_date_month_
SDBVMSource start_date_
SDBVMSource end_date_
SDBVMSource bodily_injury_incurred_
SDBVMSource property_damage_incurred_
SDBVMSource accident_death_incurred_
SDBVMSource collision_lou_incurred_
SDBVMSource comprehensive_incurred_
SDBVMSource bodily_injury_count_
SDBVMSource property_damage_count_
SDBVMSource accident_death_count_
SDBVMSource collision_lou_count_
SDBVMSource comprehensive_count_
SDBVMSource bodily_injury_severity_
SDBVMSource property_damage_severity_
SDBVMSource accident_death_severity_
SDBVMSource collision_lou_severity_
SDBVMSource comprehensive_severity_

Private Types

typedef SDBVMField inherited

Detailed Description

This class is specialized for processing an ICBC database with the following fields: "policy_start_date"; "policy_end_date"; "bodily_injury_incurred"; "property_damage_incurred"; "accident_death_incurred"; "collision_lou_incurred"; "comprehensive_incurred"; "roadstar_incurred";

The output has at least 1 value and at most 8 values (number of outputs = targetname=="ALL"?5:1 + int(use_roadstar) + int(add_claims_sum_column) + int(rescale_by_interval)). targetname can be "ALL" or one of the 5 other targets (excluding "roadstar_incurred"), or sum_all_but_BI. There are targetname=="ALL"?5:1 + int(use_roadstar) "*_incurred" targets + possibly one target corresponding to the sum of all the "*_incurred" targets (if add_claims_sum_column==true) and possibly one "weight" if rescale_by_interval==true. If targetname=="sum_all_but_BI" then there is only one target, which is the sum of all 4 KOLs excluding Bodily Injury.

The weight is computed as follows: duration = Year(policy_end_date)-Year(policy_start_date) weight = duration^2

The 5 + int(use_roadstar) targets are computed as follows: _incurred_target = 0.001 * *_incurred; if (rescale_by_interval) _incurred_target /= duration if (rescale_by_start_date) _incurred_target /= smoothed *_incurred monthly mean for month m and year y where m and y are the month and year of the policy start date of the contract. (PS: for more information on the smoothed monthly means, read the comments in PLearn/Databases/ICBCdatabases.h) if is_missing(X_incurred_target) then X_incurred_target = 0

where Year is the date given in units of years (with fractional part), so that in the common case where the difference is one year, the weight is 1 and there is no normalization of the targets. Note that the target outputs are in UNITS OF 1000 DOLLARS PER YEAR.

This "weight" is the weight that should be given to the squared loss (f(x)-normalized_target)^2. This comes from the following reasoning:

We are to learn a function f(x) that predicts the incurred amounts for ONE YEAR, but the (x,target) pairs in the data may cover more or less than one year. So the loss should be (f(x)*duration - target)^2 where duration is in units of years. To make it more "standard" we instead consider the weighted squared loss, since (f(x)*duration - target)^2 = duration^2 * (f(x) - target/duration)^2 = w * (f(x) - normalized_target)^2

When add_claims_sum_column==true sum of targets = sum of all the *_incurred targets after they have been rescaled

PS: * Year is the date given in units of years (with fractional part), so that in the common case where the difference is one year, the weight is 1 and there is no normalization of the targets. Note that the target outputs are in UNITS OF 1000 DOLLARS PER YEAR. When rescale_by_interval==true, the "weight" is the weight that should be given to the squared loss (f(x)-normalized_target)^2. This comes from the following reasoning: We are to learn a function f(x) that predicts the incurred amounts for ONE YEAR, but the (x,target) pairs in the data may cover more or less than one year. So the loss should be (f(x)*duration - target)^2 where duration is in units of years. To make it more "standard" we instead consider the weighted squared loss, since (f(x)*duration - target)^2 = duration^2 * (f(x) - target/duration)^2 = w * (f(x) - normalized_target)^2

Definition at line 971 of file SDBVMat.h.


Member Typedef Documentation

Definition at line 973 of file SDBVMat.h.


Constructor & Destructor Documentation

PLearn::SDBVMFieldICBCTargets::SDBVMFieldICBCTargets ( Schema  schema,
bool  use_roadstar,
bool  add_claims_sum_column,
bool  rescale_by_interval,
bool  rescale_by_start_date,
Mat start_date_rescaling_values,
const string &  targetname = "ALL" 
)

Definition at line 953 of file SDBVMat.cc.

References reference_start_date_month_, reference_start_date_year_, rescale_by_start_date_, and start_date_rescaling_values_.

    : inherited(0,VMField::Continuous), use_roadstar_(use_roadstar),
      add_claims_sum_column_(add_claims_sum_column),
      rescale_by_interval_(rescale_by_interval),
      rescale_by_start_date_(rescale_by_start_date),
      start_date_rescaling_values_(start_date_rescaling_values),
      targetname_(targetname),
      start_date_(schema("policy_start_date")),
      end_date_(schema("policy_end_date")),
      bodily_injury_incurred_((targetname_=="ALL" || targetname_=="bodily_injury_incurred") ?
                              schema("bodily_injury_incurred") : FieldPtr()),
      property_damage_incurred_((targetname_=="ALL" || targetname_=="sum_all_but_BI" 
                                 || targetname_=="property_damage_incurred") ?
                                schema("property_damage_incurred") : FieldPtr()),
      accident_death_incurred_((targetname_=="ALL" || targetname_=="sum_all_but_BI" 
                                || targetname_=="accident_death_incurred") ?
                               schema("accident_death_incurred") : FieldPtr()),
      collision_lou_incurred_((targetname_=="ALL" || targetname_=="sum_all_but_BI"
                               || targetname_=="collision_lou_incurred") ?
                              schema("collision_lou_incurred") : FieldPtr()),
      comprehensive_incurred_((targetname_=="ALL" || targetname_=="sum_all_but_BI"
                               || targetname_=="comprehensive_incurred") ?
                              schema("comprehensive_incurred") : FieldPtr()),
      bodily_injury_count_((targetname_=="ALLcounts" || targetname_=="bodily_injury_count") ?
                           schema("bodily_injury_count") : FieldPtr()),
      property_damage_count_((targetname_=="ALLcounts" || targetname_=="all_counts_but_BI"
                              || targetname_=="property_damage_count") ?
                             schema("property_damage_count") : FieldPtr()),
      accident_death_count_((targetname_=="ALLcounts" || targetname_=="all_counts_but_BI"
                             || targetname_=="accident_death_count") ?
                            schema("accident_death_count") : FieldPtr()),
      collision_lou_count_((targetname_=="ALLcounts" || targetname_=="all_counts_but_BI"
                            || targetname_=="collision_lou_count") ?
                           schema("collision_lou_count") : FieldPtr()),
      comprehensive_count_((targetname_=="ALLcounts" || targetname_=="all_counts_but_BI"
                            || targetname_=="comprehensive_count") ?
                           schema("comprehensive_count") : FieldPtr()),
      bodily_injury_severity_((targetname_=="ALLseverities" || targetname_=="bodily_injury_severity") ?
                              schema("bodily_injury_severity") : FieldPtr()),
      property_damage_severity_((targetname_=="ALLseverities" || targetname_=="all_severities_but_BI"
                                 || targetname_=="property_damage_severity") ?
                                schema("property_damage_severity") : FieldPtr()),
      accident_death_severity_((targetname_=="ALLseverities" || targetname_=="all_severities_but_BI"
                                || targetname_=="accident_death_severity") ?
                               schema("accident_death_severity") : FieldPtr()),
      collision_lou_severity_((targetname_=="ALLseverities" || targetname_=="all_severities_but_BI"
                               || targetname_=="collision_lou_severity") ?
                              schema("collision_lou_severity") : FieldPtr()),
      comprehensive_severity_((targetname_=="ALLseverities" || targetname_=="all_severities_but_BI"
                               || targetname_=="comprehensive_severity") ?
                              schema("comprehensive_severity") : FieldPtr())
    // roadstar_incurred_((use_roadstar && (targetname_ == "ALL" || targetname == "roadstar_incurred")) ?
    //                   schema("roadstar_incurred") : FieldPtr())

{
    reference_start_date_year_ = rescale_by_start_date_ ? (int) start_date_rescaling_values_[0][0] : 0;
    reference_start_date_month_ = rescale_by_start_date_ ? (int) start_date_rescaling_values_[0][1] : 0;
}

Member Function Documentation

void PLearn::SDBVMFieldICBCTargets::convertField ( const SDBWithStats sdb,
const Row theRow,
const Vec outputField 
) const [virtual]

Given a database row, convert the appropriate parts to a (preallocated) output vector of the correct width (given by fieldWidth). Replace MISSING_VALUEs by missing_values_mapping.

Implements PLearn::SDBVMField.

Definition at line 1014 of file SDBVMat.cc.

References accident_death_count_, accident_death_incurred_, add_claims_sum_column_, bodily_injury_count_, bodily_injury_incurred_, collision_lou_count_, collision_lou_incurred_, comprehensive_count_, comprehensive_incurred_, PLearn::SDBVMField::convertMissing(), end_date_, PLearn::endl(), PLearn::SDBVMSource::getValue(), i, if(), PLearn::is_missing(), PLearn::PDate::month, n, PLERROR, property_damage_count_, property_damage_incurred_, reference_start_date_month_, reference_start_date_year_, rescale_by_interval_, rescale_by_start_date_, start_date_, start_date_rescaling_values_, targetname_, PLearn::FieldValue::toDate(), and PLearn::PDate::year.

{
    int i = 0;
    PDate start_date = start_date_.getValue(sdb,row).toDate();
    PDate end_date = end_date_.getValue(sdb,row).toDate();
    // ******* BIG HACK TO GET AROUND A NASTY INCOMPREHENSIBLE BUG ****
    // (with the value read from file at this point not corresponding
    // to the value as seen otherwise in the file for the date of certain records)
    if (start_date.year<1970) start_date.year = end_date.year-1;
    // ****************************************************************
    real normalization = 0.001;
    real duration = (end_date - start_date)/365.25;

    if (rescale_by_interval_) {
        if (is_missing(duration) || duration<=0)
        {
            cout << "start_date = " << start_date << endl;
            cout << "end_date = " << end_date << endl;
            PLERROR("SDBVMFieldICBCTargets: incorrect dates");
        }
        normalization = 0.001/duration;
    }
    if (rescale_by_start_date_) {
        int row_index = (start_date.year - reference_start_date_year_ - 1) * 12
            + 12 - reference_start_date_month_ + start_date.month;
        if (targetname_=="ALL" || targetname_=="bodily_injury_incurred")
        {
            if (start_date_rescaling_values_[row_index][2] == 0)
                PLERROR("Trying to divide by zero");
            output[i++] = real(bodily_injury_incurred_.getValue(sdb,row))*normalization /
                start_date_rescaling_values_[row_index][2];
        }
        if (targetname_=="ALL" || targetname_=="property_damage_incurred")
        {
            if (start_date_rescaling_values_[row_index][3] == 0)
                PLERROR("Trying to divide by zero");
            output[i++] = real(property_damage_incurred_.getValue(sdb,row))*normalization /
                start_date_rescaling_values_[row_index][3];
        }
        if (targetname_=="ALL" || targetname_=="accident_death_incurred")
        {
            if (start_date_rescaling_values_[row_index][4] == 0)
                PLERROR("Trying to divide by zero");
            output[i++] = real(accident_death_incurred_.getValue(sdb,row))*normalization /
                start_date_rescaling_values_[row_index][4];
        }
        if (targetname_=="ALL" || targetname_=="collision_lou_incurred")
        {
            if (start_date_rescaling_values_[row_index][5] == 0)
                PLERROR("Trying to divide by zero");
            output[i++] = real(collision_lou_incurred_.getValue(sdb,row))*normalization /
                start_date_rescaling_values_[row_index][5];
        }
        if (targetname_=="ALL" || targetname_=="comprehensive_incurred")
        {
            if (start_date_rescaling_values_[row_index][6] == 0)
                PLERROR("Trying to divide by zero");
            output[i++] = real(comprehensive_incurred_.getValue(sdb,row))*normalization /
                start_date_rescaling_values_[row_index][6];
        }
        if (targetname_=="sum_all_but_BI")
        {
            if (start_date_rescaling_values_[row_index][3] == 0 ||
                start_date_rescaling_values_[row_index][4] == 0 ||
                start_date_rescaling_values_[row_index][5] == 0 ||
                start_date_rescaling_values_[row_index][6] == 0)
                PLERROR("Trying to divide by zero");
            output[i++] = (real(property_damage_incurred_.getValue(sdb,row))/
                           start_date_rescaling_values_[row_index][3] +
                           real(accident_death_incurred_.getValue(sdb,row))/
                           start_date_rescaling_values_[row_index][4] +
                           real(collision_lou_incurred_.getValue(sdb,row))/
                           start_date_rescaling_values_[row_index][5] +
                           real(comprehensive_incurred_.getValue(sdb,row))/
                           start_date_rescaling_values_[row_index][6])*normalization;
        }
    }
    else { //  no rescale by start date
        if (targetname_=="ALL" || targetname_=="bodily_injury_incurred")
            output[i++] = real(bodily_injury_incurred_.getValue(sdb,row))*normalization; 
        if (targetname_=="ALL" || targetname_=="property_damage_incurred")
            output[i++] = real(property_damage_incurred_.getValue(sdb,row))*normalization; 
        if (targetname_=="ALL" || targetname_=="accident_death_incurred")
            output[i++] = real(accident_death_incurred_.getValue(sdb,row))*normalization; 
        if (targetname_=="ALL" || targetname_=="collision_lou_incurred")
            output[i++] = real(collision_lou_incurred_.getValue(sdb,row))*normalization; 
        if (targetname_=="ALL" || targetname_=="comprehensive_incurred")
            output[i++] = real(comprehensive_incurred_.getValue(sdb,row))*normalization; 
        if (targetname_=="sum_all_but_BI")
            output[i++] = (real(property_damage_incurred_.getValue(sdb,row))+
                           real(accident_death_incurred_.getValue(sdb,row))+
                           real(collision_lou_incurred_.getValue(sdb,row))+
                           real(comprehensive_incurred_.getValue(sdb,row)))*normalization;

        //counts
        if (targetname_=="ALLcounts" || targetname_=="bodily_injury_count")
            output[i++] = real(bodily_injury_count_.getValue(sdb,row));
        if (targetname_=="ALLcounts" || targetname_=="all_counts_but_BI" || targetname_=="property_damage_count")
            output[i++] = real(property_damage_count_.getValue(sdb,row));
        if (targetname_=="ALLcounts" || targetname_=="all_counts_but_BI" || targetname_=="accident_death_count")
            output[i++] = real(accident_death_count_.getValue(sdb,row));
        if (targetname_=="ALLcounts" || targetname_=="all_counts_but_BI" || targetname_=="collision_lou_count")
            output[i++] = real(collision_lou_count_.getValue(sdb,row));
        if (targetname_=="ALLcounts" || targetname_=="all_counts_but_BI" || targetname_=="comprehensive_count")
            output[i++] = real(comprehensive_count_.getValue(sdb,row));


        //severities
        if (targetname_=="ALLseverities" || targetname_=="bodily_injury_severity")
        {
            int n = int(real(bodily_injury_count_.getValue(sdb,row)));
            output[i++] = n>0? real(bodily_injury_incurred_.getValue(sdb,row)) / n: 0;
        }
        if (targetname_=="ALLseverities" || targetname_=="all_severities_but_BI" || targetname_=="property_damage_severity")
        {
            int n = int(real(property_damage_count_.getValue(sdb,row)));
            output[i++] = n>0? real(property_damage_incurred_.getValue(sdb,row)) / n: 0;
        }
        if (targetname_=="ALLseverities" || targetname_=="all_severities_but_BI" || targetname_=="accident_death_severity")
        {
            int n = int(real(accident_death_count_.getValue(sdb,row)));
            output[i++] = n>0? real(accident_death_incurred_.getValue(sdb,row)) / n: 0;
        }
        if (targetname_=="ALLseverities" || targetname_=="all_severities_but_BI" || targetname_=="collision_lou_severity")
        {
            int n = int(real(collision_lou_count_.getValue(sdb,row)));
            output[i++] = n>0? real(collision_lou_incurred_.getValue(sdb,row)) / n: 0;
        }
        if (targetname_=="ALLseverities" || targetname_=="all_severities_but_BI" || targetname_=="comprehensive_severity")
        {
            int n = int(real(comprehensive_count_.getValue(sdb,row)));
            output[i++] = n>0? real(comprehensive_incurred_.getValue(sdb,row)) / n: 0;
        }

        //severity weights
        if (targetname_=="ALLseverities")
            output[i++] = real(bodily_injury_count_.getValue(sdb,row))>0? 1 : 0;
        if (targetname_=="ALLseverities" || targetname_=="all_severities_but_BI")
        {
            output[i++] = real(property_damage_count_.getValue(sdb,row))>0? 1 : 0;
            output[i++] = real(accident_death_count_.getValue(sdb,row))>0? 1 : 0;
            output[i++] = real(collision_lou_count_.getValue(sdb,row))>0? 1 : 0;
            output[i++] = real(comprehensive_count_.getValue(sdb,row))>0? 1 : 0;
        }

    }
    if (add_claims_sum_column_) {
        if (targetname_=="ALL")
            output[i] = output[0] + output[1] + output[2] + output[3] + output[4];
        else
            output[i] = output[0];
        i++;
    }
    if (rescale_by_interval_)
        output[i] = duration; // weight that should be given to squared loss
    // REPLACE MISSING VALUES BY ZEROS
    convertMissing(output);
}

Here is the call graph for this function:

virtual int PLearn::SDBVMFieldICBCTargets::fieldWidth ( ) const [inline, virtual]

Width that the field occupies in the VMat.

Implements PLearn::SDBVMField.

Definition at line 983 of file SDBVMat.h.

                                   {
        if (targetname_=="ALL")
            return 5+int(use_roadstar_)+int(add_claims_sum_column_)+int(rescale_by_interval_); 
        else if (targetname_=="sum_all_but_BI")
            return 1;
        else if (targetname_=="ALLcounts")
            return 5;
        else if (targetname_=="all_counts_but_BI")
            return 4;
        else if (targetname_=="ALLseverities")
            return 10;
        else if (targetname_=="all_severities_but_BI")
            return 8;
        else
            return 1+int(use_roadstar_)+int(add_claims_sum_column_)+int(rescale_by_interval_);}

Member Data Documentation

Definition at line 1018 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1012 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1024 of file SDBVMat.h.

Definition at line 1001 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1016 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1010 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1022 of file SDBVMat.h.

Definition at line 1019 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1013 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1025 of file SDBVMat.h.

Definition at line 1020 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1014 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1026 of file SDBVMat.h.

Definition at line 1009 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1017 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1011 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1023 of file SDBVMat.h.

Definition at line 1007 of file SDBVMat.h.

Referenced by convertField(), and SDBVMFieldICBCTargets().

Definition at line 1006 of file SDBVMat.h.

Referenced by convertField(), and SDBVMFieldICBCTargets().

Definition at line 1002 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1003 of file SDBVMat.h.

Referenced by convertField(), and SDBVMFieldICBCTargets().

Definition at line 1008 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1004 of file SDBVMat.h.

Referenced by convertField(), and SDBVMFieldICBCTargets().

Definition at line 1005 of file SDBVMat.h.

Referenced by convertField().

Definition at line 1000 of file SDBVMat.h.


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines