Getting started

ScoringRules.jl evaluates probabilistic forecasts using proper scoring rules. This page walks through the most common workflow — scoring a parametric or ensemble forecast against an observation — and points to the deeper guides.

Installation

using Pkg
Pkg.add("ScoringRules")

A first example

using ScoringRules, Distributions

# Score a Normal forecast against an observation
d = Normal(0.0, 1.0)
y = 0.8

crps(d, y)   # continuous ranked probability score

0.47622488424177456

logs(d, y)   # logarithmic score (= −log-likelihood)

1.238938533204673

dss(d, y)    # Dawid–Sebastiani score

0.6400000000000001

All three functions follow the lower-is-better (negative-orientation) convention throughout: a smaller score indicates a better forecast.

Ensemble forecasts

When a distributional forecast is not available but simulation draws are, pass the sample vector directly:

draws = randn(500)         # 500 samples from the forecast
crps(draws, y)

0.46086776048598055

dss(draws, y)

0.6256196588967745

Broadcasting over many forecasts

Use Julia's dot syntax to score a vector of forecasts against a matching vector of observations:

ds = [Normal(0.0, 1.0), Normal(1.0, 2.0), Normal(-0.5, 0.5)]
ys = [0.8, 1.2, -0.3]

crps.(ds, ys)

3-element Vector{Float64}:
 0.47622488424177456
 0.4753621577233462
 0.14834404517357486

mean(crps.(ds, ys))   # mean score over the evaluation set

0.3666436957128985

Count and ordinal forecasts

The same interface handles discrete distributions:

logs(Poisson(3.0), 2)

1.4959226032237258

crps(NegativeBinomial(5, 0.4), 3)

2.421010059117793

Learning more

• Forecast input modes — parametric, ensemble, moment-based, and quantile inputs.

• Scoring rules reference — multivariate, weighted, quantile, and ordinal scores.

• Supported distributions — every family with a closed-form CRPS.

• Differences from R — known divergences from the R scoringRules package.

• Public API — complete function reference.

Attribution

ScoringRules.jl is a Julia port of the R package scoringRules by Alexander I. Jordan, Fabian Krüger, Sebastian Lerch and Sam Allen. The initial port was generated by a large language model (Claude) under human guidance. See the package README for full attribution and licence details.