DIMACS Workshop on Forecasting: From Forecasts to Decisions

March 17, 2021 - March 19, 2021

Location:

Online Event

Organizer(s):

Raf Frongillo, University of Colorado

David Pennock, DIMACS

Bo Waggoner, University of Colorado

Description

Following the successful EC 2017 Workshop on Forecasting, we will hold the DIMACS Workshop on Forecasting in 2021. We welcome submissions describing recent research on crowd-sourced, data-driven, or hybrid approaches to forecasting. We especially encourage contributions that leverage forecasts to improve decisions. Please see the Call for Participation below for details.

Recent advances in crowdsourced forecasting mechanisms, including Good Judgment’s superforecasting, prediction markets, wagering mechanisms, and peer-prediction systems, have risen in parallel to advances in machine learning and other data-driven forecasting approaches. Innovations have come from academic researchers, companies, data journalists, and government programs like IARPA’s Aggregative Contingent Estimation program and Hybrid Forecasting Competition.

The workshop will emphasize forecasts embedded inside decision-making systems, where the value of a forecast comes from increasing the expected utility of a key decision. Our ultimate goal is to modernize organizations, markets, and governments by improving how they collect and combine information and make decisions.

The workshop embraces the diversity of this exciting and expanding field and encourages submissions from a rich set of empirical, experimental, and theoretical perspectives. We invite theoretical computer scientists studying algorithmic game theory, incentivized exploration, and NP-hard counting problems; AI researchers studying machine learning, human computation, Bayesian inference, peer prediction, and satisfiability; statisticians studying scoring rules and belief aggregation; economists studying prediction markets, financial markets, and wagering mechanisms; data journalists and marketing scientists studying surveys and polls; blockchain pioneers implementing decentralized prediction markets and other experimental market constructs; social and behavioral scientists studying human behavior modeling; human-computer interaction researchers designing interfaces to facilitate elicitation or convey uncertainty; and practitioners working to improve forecasts as a business or service.

Uncertainty is hard to communicate. Forecasters argue that they are “right”, and critics that forecasters are “wrong” (for example about Brexit or the US Presidential election), despite the fact that probabilistic forecasts can only be evaluated in bulk relative to other forecasts. We invite contributions discussing ways to communicate uncertainty and educate the public about modeling, forecasting, and scoring, building on the excellent 2018 Nova episode “Prediction by the Numbers”.

Topics of interest for the workshop include but are not limited to:

Incentives in forecasting. Methods for eliciting truthful and accurate forecasts or information.
Coordinating groups of participants to collectively forecast. Examples include prediction markets and wagering mechanisms.
Connections between human- and machine-driven forecasting. Uses of data, models, or machine learning in forecasting, and theoretical connections between forecasting mechanisms and machine learning techniques.
Making complex forecasts. Predicting structured, combinatorial, or multi-part events. Making conditional forecasts. Forecasting continuous distributions, exponential-sized joint distributions, and spatiotemporal distributions.
Forecasting metrics related to climate, the environment, transportation, renewable energy, or public health. For example, metrics of a pandemic including number infected, number hospitalized, number killed, and fatality rate by region and over time, conditioned on public health policies.
Forecasting in support of decision making by companies, organizations, or governments.
Visualization and other best practices for communicating uncertainty and educating the public about forecasts.

Please note times specified in program below are Eastern Time.


Program

Wednesday, March 17, 2021
Session - 1 - Chair: Bo Waggoner, University of Colorado
10:00 AM - 10:10 AM

Welcome & Opening Remarks
10:10 AM - 10:45 AM

Invited Talk: How to Increase the Accuracy of Human Forecasts and Check the Reasons for Improvement

Barbara Mellers, University of Pennsylvania

Ville Satopää, INSEAD
10:45 AM - 11:05 AM

Asymptotic Behaviour of Prediction Markets

Philip Dawid, University of Cambridge
11:05 AM - 11:25 AM

Timely Information from Prediction Markets

Chenkai Yu, Tsinghua University
11:25 AM - 11:35 AM

Break
11:35 AM - 12:10 PM

Invited Talk: A Heuristic for Combining Correlated Experts

Yael Gruska-Cockayne, University of Virginia
Session - Poster Session 1
12:10 PM - 1:00 PM

View Posters & Videos


Thursday, March 18, 2021
Session - 2 - Chair: Raf Frongillo, University of Colorado
10:00 AM - 10:35 AM

Invited Talk: Predicting Replication Outcomes

Anna Dreber, Stockholm School of Economics
10:35 AM - 10:55 AM

From Proper Scoring Rules to Max-Min Optimal Forecast Aggregation

Eric Neyman, Columbia University
10:55 AM - 11:15 AM

Forecast Aggregation via Peer Prediction

Juntao Wang, Harvard University
11:15 AM - 11:35 AM

Comparing Forecasting Skill vs Domain Expertise for Policy-Relevant Crowd-Forecasting

Emile Servan-Schreiber, Mohammed VI Polytechnic University
Session - Panel - Moderator: David Pennock, DIMACS
11:35 AM - 12:10 PM

Forecasting Startup Founders Panel

Pavel Atanasov, pytho

Andreas Katsouris, PredictIt

Kelly Littlepage, OneChronos

Emile Servan-Schreiber, Hypermind
Session - Poster Session 2
12:10 PM - 1:00 PM

View Posters & Videos
1:00 PM - 1:30 PM

Social Event


Friday, March 19, 2021
Session - 3 - Chair: David Pennock, DIMACS
10:00 AM - 10:35 AM

Invited Talk: Information, Incentives, and Goals in Election Forecasts

Andrew Gelman, Columbia University
10:35 AM - 10:55 AM

Models, Markets, and the Forecasting of Elections

Rajiv Sethi, Columbia University
10:55 AM - 11:15 AM

Boosting the Wisdom of Crowds Within a Single Judgment Problem: Weighted Averaging Based on Peer Predictions

Ville Satopää, INSEAD
11:15 AM - 11:35 AM

Crowdsourced Forecast Elicitation: Methods vs. Individuals

Pavel Atanasov, pytho
11:35 AM - 12:10 PM

Invited Talk: Models vs. Markets: Forecasting the 2020 U.S. election

Harry Crane, Rutgers University
Session - Poster Session 3
12:10 PM - 1:00 PM

View Posters & Videos


Call For Participation

Attend: This workshop is open to all to attend, but you must register using the link at the bottom of the page. We will send instructions on how to join the event on or before March 15, 2021. If you do not receive them, please check your spam folder or contact Nicole Clark. Please note that you may not be able to register once the event has begun.



Present: We invite both full contributions and poster contributions. A full contribution is an unpublished or recently published research manuscript. A poster contribution can be a preprint, a recently published paper, an abstract, or a presentation file. Preference may be given to more recent and unpublished work. We especially encourage poster contributions from students and postdocs.



Please submit your contributions using this Google Form by February 19, 2021. The workshop is non-archival, meaning contributors are free to publish their results later in archival journals or conferences. Panel discussion proposals and invited speaker suggestions are also welcome. Email questions or suggestions to the organizers.



The workshop will include invited and contributed talks, open discussion, and may include a poster session and a rump session. Workshop registration will be open. Once registered, you will join the workshop through Virtual Chair.



Important Dates:

Submissions due: Friday, February 19, 2021
Notifications: Wednesday, March 3, 2021
Workshop: March 17-19, 2021

Presented in association with the SF on Mechanisms & Algorithms to Augment Human Decision Making.