Regression/Classification metrics optimization

https://www.coursera.org/learn/competitive-data-science/lecture/SQ9Uq/regression-metrics-optimization

MAE (L1 metric/L1 loss) - some methods don't support this metric since second derivative is zero.
RMSLE

Classification metrics optimization

https://www.coursera.org/learn/competitive-data-science/lecture/hvDC5/classification-metrics-optimization-i

logloss - very popular(like MSE for regression) - all NNs by default optimize logloss for classification. RFs turn out to be very bad in terms of logloss but they can be made better.

logloss requires models to output posterior probabilities - but what does it mean? logloss is easy to implement.

accuracy -

installing dropbox on centos linux

yum install docker - y
alias dropbox="docker exec -it dropbox dropbox"
alias dropbox-start="docker run -d --restart=always --name=dropbox -v
/home/val/Dropbox:/dbox/Dropbox -v /home/val/.dropbox:/dbox/.dropbox
-e DBOX_UID=1000 -e DBOX_GID=100 janeczku/dropbox"
sudo systemctl start docker
dropbox-start
docker ps (will give you the pid for e.g. a6f6a1d0866f)
docker logs --follow a6f6a1d0866f (now you will see a web link to
connect dropbox to your pc)

metrics and loss function

Sometimes both are same for e.g. MSE. But at times they will be
different since it is hard to define loss function for some metrics.
In XgBoost you can write cusotm loss functions but they should have
smooth derivatives otherwise it will go crazy.

Early stopping - In case we don't know how to write a loss function
for the target metric, there is a simple solution - Early Stopping.
Keep optimizing the loss function but stop when the model starts over
fitting as per the target metric.

area under curve (AUC) simple explanation

Best explanation I have found is here:
https://www.coursera.org/learn/competitive-data-science/lecture/EhJzY/classification-metrics-review

1) Area under curve
For every threshold plot number of TPs on y-axis and FPs on x-axis. If
dataset can be clearly separated by a threshold then the AUC will be
1(max value).

2) Pair ordering
Consider all possible pairs such that one item is TP and another is
FP. AUC is the probability that the FP is ranked higher than TP (so
that threshold puts TPs on the left and FPs on the right).

Classification metrics

Accuracy
logloss, hard predictions, soft predictions
AUC?
Cohen's Kappa - error weight matrix - linear weighted or quadratic
weighted kappa.