THE ST. petersburg paradox

• an actual thing that people used to do: gambling based on results of single flipped coins.
• stake starts at 2 bucks, doubles on every tails, you win if heads.
• how much is a fair price to pay in to the game, and why isn't your answer INFINITY DOLLARS

Edge cases

• Evidential & Causal Decision Theories
• Traditional DT is "Causal", i.e. makes the decision with the best expected causal consequences
• This falls into a fairly silly trap, though:

https://wiki.lesswrong.com/wiki/Parfit's_hitchhiker

"Suppose you're out in the desert, running out of water, and soon to die - when someone in a motor vehicle drives up next to you. Furthermore, the driver of the motor vehicle is a perfectly selfish ideal game-theoretic agent, and even further, so are you. The driver says, "Well, I'll get you to town if it's in my interest to do so - so will you give me $100 from an ATM when we reach town?"

Now of course you wish you could answer "Yes", but as an ideal game theorist yourself, you realize that, once you actually reach town, you'll have no further motive to pay off the driver. "Yes," you say. "You're lying," says the driver, and drives off leaving you to die.

If only you weren't so rational!"

Evidential DT

• The best action is the one which, conditional on your having chosen it, gives you the best expectations for the outcome.
•  Also has a silly failure mode:

https://wiki.lesswrong.com/wiki/Smoker's_lesion

Smoking is strongly correlated with lung cancer, but in the world of the Smoker's Lesion this correlation is understood to be the result of a common cause: a genetic lesion that tends to cause both smoking and cancer. Once we fix the presence or absence of the lesion, there is no additional correlation between smoking and cancer.

Suppose you prefer smoking without cancer to not smoking without cancer, and prefer smoking with cancer to not smoking with cancer. Should you smoke?

Causal says "yes", evidential says "no".

I'm Omega - an omniscient, godlike superintelligence capable of predicting your actions with 100% accuracy.

Either only open A, or open A and B.

 I put $1,000 in box B.

I put $1,000,000 in box A if I think you'll open box A and only box A.

How many boxes do you want to open? :3