날짜: 23년 10월 11일 (수) 시간: 16:00-17:30 주최: 연세대학교 상경대학 위치: 연세대학교 대우관 각당헌(B130) 연사: John A. List (Author of ‘The Voltage Effect’)

Lecture

• Always ask yourself: will your (new) idea ‘scale’?

  • The ‘science’ of scaling, NOT ‘art’

• Cf. In 2008, a community called Chicago Height called John for help

  • A broken-down community…
  • John built his own pre-K school
    • target: 3-5 yrs old
    • goal: build a curriculum that can scale
    • response: the result seems impressive, but don’t expect it to happen at scale.
      • reason: all of the experts told us that their intervention would work, but it didn’t.

• Cf. Throughout his career, John has been scaling

  • Ex. White House (would the policy scale?), Uber, etc.

• Why not add some more “science” in the art of scaling?

  • Empiricists focus on how best to generate data to show intervention effects & mechanisms
  • John: we should also ask whether the idea is scalable at the initial stage

• The best case scenario of publishing your academic paper: two people read it; the editor and one of the three referees (ㅋㅋㅋㅋ)

  ∵ requires background knowledge

  → John: I’ll write in a way that my father (a truck driver) can understand it

• The 1st Law of Scaling:

  • Voltage Effect: as you move from the small to the big, effect sizes change
    • who would buy a book called “the wattage effect”? give me some artistic liberty here (ㅋㅋㅋㅋ)

• The 2nd Law of Scaling:

  • This is not about a “silver bullet”: idea must possess the 5Vital signs before one can have scaling confidence

    • These 5Vitals provide insights into breadth and depth of your policy (cf. “The Anna Karenina problem”)

  • 5Vitals:

    1. False Positives

       Inference problem

       1. statistical error (alpha)
       2. human error (confirmation bias, etc.)
       3. human fraud (cf. “fake it till you make it”)

       cf. Once you have evidence, how much weight should you put unto that evidence?

       ⇒ More replications → higher probability

       • How false positives could be addressed

       Ex. Nancy Reagan’s “Just Say No” campaign (D.A.R.E. program)

       • it was a false positive

    2. Know your audience

       Horizontal Scaling (=”Spread”)

       Ex. Smart thermostat (reduces CO2 emission)

       • When tested (for 12 months), it had NO effect
         • The intended end-user =/= The true end-user
         • Uses only default setting (”Smart Technology, Dumb Humans”)

    3. Chef or Ingredients?

       Initial success depends on the chef ← unscalable

       Understand what is the secret sauce of success (before trying scaling)

       • Representative Situation?

         Horizontal Scaling meets Vertical Scaling

         Humans don’t scale well; hiring 30,000 teachers rather than 30 is very different

       • Efficacy test in social sciences is a wrong approach

         • Add ‘option C’ to ‘A/B’ test
           • hire really bad teachers along w/ good teachers
           • option C = policy-based evidence
           • & situationally-congruent evidence

    4. Spillover

       Ex1. A kid raised w/in the vicinity of 3 other kids under treatment → similar result

       Ex2. #Deleteuber campaign

       • John’s goal: get drivers back engaged!
         • Solution: introduce tipping!
           • experiments on 5% of drivers: the drivers earned more & worked more
           • on 100% of drivers: more drivers came → more supply → no change in earning

    5. Supply side of scaling

    (⇒ ofc, we have to test them empirically.)

• only 1% of people tipped on every trip

  • 60% of people never tipped
    • when it’s face-to-face, over 90% tipped (social pressure, etc. is a social incentive that scales very well)

• Economists think on the margins

  • Ex. recruiting 1,000 drivers via…
    • FB: $300 on aver. ($1,000 for the last 25)
    • Google: $400 on aver. ($500 for the last 25)
    • Always slice your data in a finer manner

• Always recognize opportunity loss

  • Until something bad happens, we don’t think about opportunities

QnA

• Experience helps you to come up w/research thesis.
• Empirically, tipping doesn’t improve service quality (it is welfare-reducing, and it makes employers pay less for employees)
• In Economics, replication doesn’t happen often.
  • Suggestion1: let’s do independent replications and make a joint-publication
  • Suggestion2: make a journal where replications are published
    • For Economics to be truly science, its findings should be replicable & done so.
    • Null results must be appreciated as much as positive results (← Bayesian improvement)