Saturday, March 25, 2017

Beyond Gen II

This is my own note for how to prepare for future generations, based on two references: one and two.

Generation III (nothing to prepare for)

Babies (requires hatching):
Azurill (Marill baby)
Wynaut (Wobbuffet baby)
Non-babies (can be caught in wild):
Ralts> Kirlia > Gardevor (25, and 100 candies)
Nosepass
Roselia
Duskull > Dusclops (50 candies)
Chimecho
Snorunt > Glalie (50 candies)
Clamperl > Gorebyss (50 candies, evo item: deepseascale)
Clamperl > Huntail (50 candies, evo item: deepseatooth)
Feebas > Milotic (50 candies, evo item: prism scale, item introduced in Gen5?)

Generation IV (Need to save candies)

Babies (require hatching):
Happny (Chansey baby)
Mime Jr. (Mr. Mime baby)
Munchlax (Snorlax baby)
Bonsly (Sudowoodo baby)
Mantyke (Mantine baby)
Chingling (Chimecho (gen 3) baby) 
Evolutions from previous generation:
Magneton > Magnezone (100 candies, no evo item)
Lickitung > Likilicky (50 candies, evo item: rollout)
Rhydon > Rhyperior (100 candies, evo item: protector)
Tangela > Tangrowth (50 candies, evo item: ancient power)
Electabuzz > Electivire (50 candies, evo item: electrizer)
Magmar > Magmortar (50 candies, evo item: magmarizer)
Eevee > Leafeon (25 candies, evo item: moss rock)
Eevee > Glaceon (25 candies, evo item: ice rock)
Porygon2 > Porygon-Z (most likely 50 candies from Porygon instead of Porygon2 based on the past pattern, evo item: dubious disc)
Togetic > Togekiss (most likely 50 candies from Togepi instead of Togetic based on past pattern, evo item: shiny stone)
Aipom > Ambipom (50 candies, evo item: double hit)
Yanma > Yanmega (50 candies, evo item: ancient power)
Murkrow > honchkrow (50 candies, evo item: dusk stone)
Misdreavus > Mismaguis (50 candies, evo item: dusk stone)
Gilgar > Gliscor (50 candies, evo item: razor fang)
Sneasel > Weavile (50 candies, evo item: razor claw)
Piloswine > Mamoswine (most likely 100 candies from Swinub based on past pattern, evo item: ancient power)
Kirlia (Gen3) > Gallade (100 candies, evo item: dawn stone)
Nosepass (Gen3) > Probopass (50 candies)
Roselia (Gen3) > Roserade (100 candies, evo item: shiny stone)
Dusclops (Gen3) > Dusknoir (100 candies, evo item: reaper cloth)
Snorunt (Gen3) > Froslass (50 candies, evo item: dawn stone)

Generation V
Non-babies (can be caught in wild):
Cottonee > Whimsicott (50 candies, evo item: sun stone)
Lampent > Chandelure (50 candies, evo item: dusk stone)
Petilil > Lilligant (50 candies, evo item: sun stone)

Generation VI

Non-babies (can be caught in wild):
Helioptile > Heliolisk (50 candies, evo item: sun stone)
Doublade > Aegislash (50 candies, evo item: dusk stone)
Spritzee > Aromatisse (50 candies, evo item: sachet)
Swirlix > Slurpuff (50 candies, evo item: whipped dream)

Evolutions from previous generation:
Eevee > Sylveon (25 candies, evo item: affection, fair-type move?)

Generation VII

Evolutions from previous generation:
Pikachu > Raichu* (50 candies, a different Raichu, evo item: thunder stone)
Exeggcute > Exeggutor* (50 candies, a different Exeggutor, evo item: leaf stone)
Cubone > Marowak* (50 candies, a different Marowak, at night, no evo item)

Friday, March 10, 2017

The Secret of Egg Hatching

The mystery behind egg-hatching has been unveiled by Silph Road researchers (here) [1]!

For a long time, the understanding has been a Pokestop picks a 2km/5km/10km egg based on an egg-probability distribution, then pick a Pokemon within that distance according to another probability distribution.  For example, the chance of getting a Lapras is $p(Lapras) = p(10km)*p(Lapras|10km)$.  Now there is a much simpler and elegant new theory: eggs are simply determined by one probability distribution - one that is associated with Pokemon alone.  Hatchable Pokemons are assigned to four categories: Common, Uncommon, Rare and Ultra Rare, with hatching probability ratio 8:4:2:1.

The assignment of Pokemons based on their hatch distance and rareness is shown in the table below [2]:

Converting this chart to counts:

RarenessWeight 2km5km 10kmTotal by Rareness (weighted),
Row Sum
Common86102144
Uncommon416153136
Rare214316
Ultra-Rare1113519
Total by Distance (weighted),
Column Sum

11516139315

Imagine there are a total of 315 eggs in a basket, Lapras (ultra-rare) has only one egg, Geodude (common) has eight eggs. Pokestop randomly picks an egg from this basket, distance (egg color) is just a by-product. Then the probability of hatch a particular Pokemon can be calculated based on its category (regardless of its egg distance):
common: 8/315 = 2.54%,
uncommon: 4/315 = 1.27%,
rare: 2/315 = 0.63%,
super rare: 1/315 = 0.32%.

The egg drop probability from a Pokestop can also be explained:
2km: 115/315 = 36.5%,
5km: 161/315 = 51.1%,
10km: 39/315 = 12.4%.
For Lapras, the chance of getting a 10km egg is 12.4%, the chance of that 10km egg being Lapras is: 1/39 = 2.56%.  The overall probability of hatching a Lapras is 1/315 = 0.32%.  This is quite consistent with our previous observations (see my previous Blog entry on hatching).  We can further infer that to hatch a Lapras with a 50% chance, one should hatch 218 eggs, including 27 10km eggs.  To hit an 80% chance, one should hatch 506 eggs, including 62 10km eggs!

Let us check on Gen 2 babies. Babies are all uncommon with a probability of 1.27%, despite of their distances.  So to hatch a particular baby, one needs to hatch 54 eggs (50% chance) or 126 eggs (80% chance) (I hatched Tyrogue with 108 eggs, in agreement).  Pick Tyrogue in particular, one should walk 25 or 59 5km eggs in order to hit 50% and 80% chance, respectively.  Although these many eggs would probably cost you $50 in incubator purchase, it is still way more feasible than hatching a Lapras.

This is indeed a beautiful theory that explains all the mysteries behind eggs! Congratulations to the researchers!

Reference

[1] https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=8&cad=rja&uact=8&ved=0ahUKEwiBroGQzczSAhXGjFQKHZckDJoQqUMIMjAH&url=https%3A%2F%2Fthesilphroad.com%2Fscience%2Fsecret-egg-rarity-tiers-pokemon-go&usg=AFQjCNHhNlqpvW0rLfDNP8byWlLfOhPA5A&sig2=6hWcKtL7eeiMm71pxKNA7w
[2] https://www.reddit.com/r/TheSilphRoad/comments/5ylpzj/chart_of_egg_rarity_divided_by_egg_distance/

Tuesday, February 21, 2017

Gen II - Catch Them All

last updated on April 6, 2017 (Gen II Complete)

Key Gen 2 Milestones:
Apr 6, 2017: Dragon Scale obtained from a spin right after 7-day streak! Complete!
Mar 30, 2017: Up-Grade obtained via 7-day streak.
Mar 21, 2017: Niantic reinstates gauranteed evolution item at 7-day streaks. Counting down to the completion of Gen 2 Pokedex;.
Mar 4, 2017: Tyrogue Hatched! Balboa Park Totodile nest, earn candies for Feraligatr.
Mar 1, 2017: Unown Found!
Feb 26, 2017: Coronado Island Hunt.
Feb 25, 2017: Girafarig Nest and Hunt in Carlsbad.
What are Gen 2s?

Source here:


Babies: Only from Eggs

They are not in wild, the only way to get babies is to hatch.  Even if you got all seven babies during the Christmas event, there is one more baby, Tyrogue, needs to be added to your Pokedex.  So buy incubators and start walking.  The secrete of hatching has been cracked recently (See my other blog entry), Tyrogue and all other babies are as hard/easy to hatch as Chammander, Bulbasaur, Oddish, or Larvitar (with 1.27% chance).  In contrast, Porygon and Lapras are 4 times harder!  So hatching a Tyrogue is definitely can be achieved.  In addition, hatching is an economic way of getting high IV and high CP pokemons, you will have to catch a lot in wild in order to get such pokemons, wasting lots of gas and eventually probably spending more money.  Every 54 eggs give you 50% chance to hatch a Tyrogue, not bad at all.
Tyrogue was finally hatched on Mar 4, 2017.  I walked 760km and hatched 489 eggs right before Gen 2 started (see blog entry), which means it took me 100km and 108 eggs to get Tyrogue.  Based on the "hatch secrete", 36.5% eggs were 2km eggs, to which I always applied the free incubator, the total cost is 108*0.63.5*$0.50/hatch = $34.29.  However, this also covers some other high-IV hatches, such as Mantine, Larvitar, Dratini, Mareep, Gilgar, Aerodactyle, Stander, Phanpy, Wooper, Slugma, etc.

Egg chart here:
Gen 2 starters are not hatchable: Chikorita, Cyndaquil, Tododile, which is actually good news, as they have nests and their removal increases the chance for hatching others.

Rares: Location, Location, Location

Remember our Gen I lesson, some Pokemons only favor coastal areas, so it is important that we visit places such as Coronado Island, La Jolla Cove, Oceanside Pier, etc. in San Diego.  Coastal-friendly ones include Lantun, Mareep, Magcargo, Forrestress, Croconaw, Bayleef, Remoraid, Quilfish, etc. However, many of these can be evolved, e.g., Forrestress can be evolved, where Pinco is not hard to get, Croconaw should be evolved from Totodile, Magcargo from Slugma, Lantun from Chinchou (seems to be missing in the egg chart), Bayleef from Chikorita, so do not be lazy here.  (The water event starts on Mar 22 certainly is a blessing for those who do not live by the beach, best of all, you get a chance to fill that empty Lapras slot!)

Nests: All Starters Have Nests

Make use of Silph Road Nest Atlas, which is pretty accurate now (link here).  I got my Girafarig at the Alga Norte Community Park (maybe 10 spawns per hour), my Sneasel at 4S Ranch Heritage Park (maybe 3 spawns per hour).  Balboa park is an extremely strong Totodile nest, it is mostly easier to catch enough candies to evolve Feraligatr yourselve than counting on encountering one in wild (feraligatr seems to be specific to coastal regions).
Girafarig nest at Algo Norte Community Park.

Balboa Park is an extremely strong Totodile nest.  You should be able to get enough candies for Feraligatr within two hours (hint: do not forget to use the Pinap Berry on weak CP ones to magnify your candy bonus)! There are 2-3 Totodile within nearby sightings constantly, take advantage of it before nest change.  Balboa Park changed into a Gastly nest on the 16th nest change, so when you see a nest of interest, act decisively.


Evolved Forms: Those Require Items are NOT in Wild

My data analysis shows any Gen II that require items (stones, steel case, dragon scale) will not exist in wild. This now has been confirmed by other analyses.  So the only way to get the following eight Pokemons is by obtaining items and evolving them yourself:  Porygon2, Bellossom, Slowking, Steelix, Kingdra, Scizor, Politoed, as well as Sunflora. Umbreon and Espeon are also not in wild.  Blissey and Corbat are in wild, as their evolution does not require items.  Items are rare from Pokestop spins, however, it seems the Day-7 Streak always give an item (happened to me as well) (Update: Evolution items from day-7 Streaks were later removed by Niantic and now reintroduced on March 21!  Without this bonus feature, it was estimated that one evolution items can be expected for every 250 Poke Stop spins, which was terrible for rural player.) So given enough time, these Pokemons are basically in your pocket, if you have played Gen I well.  Make sure get a good IV Eevee, before triggering the Umbreon and Espeon evolution.  (I have got a total of nine items so far, including one 7-day streak item, but I still need dragon scale and Up-grade.)

Items requires for Gen 2 evolution: here
Original Post here

Great news came on Mar 21, 2017!

Items requires for future generations: here (basically only 3 extra sun stones are needed)

April 6 is my 7-day streak day.  With great hope, I spinned and got a King Stone (8:33AM), how disappointing.  This is my 16th items.  I immediately did the math, the probability of an item is either Up-Grade or Dragon Scale is 1/8, the other three items are twice more likely, 1/4 each (based on some statistics on SilphRoad).  So the chance for not getting Dragon Scale 16 times on a roll is  $\left(\frac{7}{8}\right)^{16}=0.12$.  I was not super unlucky, but definitely disappointed.  As I clicked my Pogo+ on my way to work to collect some XP points, the first PokeStop I spin after getting off the highway (this is the spin right after my 7-day spin) was a Dragon Scale!  What a nice way to end my Gen II collection. 


The last wanted item - Dragon Scale was obtained with a drama.

Super Rares: Social Media & Crowd Sourcing

The chance of encountering super rares is very low by your own, so you need to read social media and find spawn information shared by other players or other sources to maximize your chance.  Still remember how hard it was to catch a Lapras, this will apply to some Pokemons in Gen II.

Here is a tentative list of super rares: Meganium (like Venusaur), Typhlosion (like Charizard), Feraligatr (like Blastoise), Togetic (as hard to capture as Dragonite), you might have a better chance to hatch its baby Togepi and walk the candies (well, I have not hatched Togepi after 100 Gen 2 eggs, so not easy either), Ampharos, Jumpluff, Forrestress (better evolve it from Pinco), Hitmontop, Tyranitar (like Dragonite, very rare, seems to favor coastal region, but Lavitar spawns universally, unlike Dratinit.). [hint: Feraligatr can only be found in coast.  Tyranitar can be found in inland.]

Again, I would really recommend evolve Meganium, Typhlosion, Feraligatr, Jumpluff and Tyranitar yourselves, instead of hoping to catch a one in wild.  Not only the chance is low, but also the likelihood of catching both a good IV and high CP one is much more rare and requires many many rare catches.  Is it not too hard to evolve one for them.  Ampharos may be hard given how rare Mareep is, some buddy-walking may be required. You may have a better chance to catch a Hitmontop than hatch two Tyrogue (save one, and use one for evolve).

Region Exclusives: Corsola and Heracross.  Tropical and South American, respectively.  Just visit enough gyms to collect them their silhouette.  The best strategy is to not let them bother you.
Corsola and Heracross were met in gyms.

Unown: Too Hard to Catch

Unowns (a-z) 26 alphabet. After one week of Gen 2, we finally see two spawns in Coronado.  It is much more rare than Lapras, so I would say forget about them, unless you live close to Coronado.  Unown is so rare that there is no point in visiting Coronado just to hit the jackpot, you most likely won't be able to.  We can only hope Niantic will increase its spawn rate or introduce future Unown events.

PS: Unown is super super rare, but there is always someone who wins the lottery.  There have been four Unowns reported so far, one in La Jolla Shores, two in Coronado Island, one in Strand (also Coronado Island).
An Unown was repored at La Jolla Shores on Mar 1, 2017.  Nearly had to get into water to catch.


Enough Unowns have been reported and there is absolutely no pattern in where they spawn.  They show up in places where normally no player would be around!  I do know a guy in Houston, who had an Unown encounter purely by lucky (without using any trackers), the majority of Unown encounters were assisted by trackers (how likely it is you happen to be at a small street at 9:30pm, and that success repeated several times?)  If you really want Unown, the only feasible way is to use your local Tracker, particularly those channels that are specifically created for Unown, so that you get alerts the moment it is broadcast.  Due to the wide spread of possible Unown spwan spots, every second counts in order to arrive at the spot on time.  Even with trackers, Unown hunting is so difficult that every Unown caught will be an unforgettable memory.  People who caught multiple Unowns in multiple cities/continents are very likely doing GPS spoofing, and they use a global Unown tracker (I do not want to name it), just don't let that bother you (sicne not all trackers report Unown letter, even you do not spoof, this tracker is useful to find out what Unown letter is spawned.  Usually spoofers will report the letter minutes after its detection, so that it may save you an unnecessary full trip.).  I forgot to mention that there are 28 Unowns (26 letters plus "!" and "?", only the 26 letters are released in Gen 2).


The Unown family.  Although Unown is only counted as one Pokedex entry, it has 28 difference manifestations.  While most players were praying for just one, some are hunting for all 26 alphabets using a global tracker and GPS spoofing tricks.


A minute after the super super-rare Unown Y was spawn in London, multiple players celebrated their completion of the Unown Gold Medal for collecting all 26 letters!  Do you want to go that far for Unown?

More than a dozen Unown-hunting vehicles showed up at a small street for Unown (I). Residents were probably shocked! 

San Diego Unown map based on 23 spawns.  Except Coronado island is an Unown hot spot, Unown spawn has been spreaded all over San Diego.


Good Candies to Save:

According to a Silph Road post: Togetic, Murkrow, Misdreavus, Aipom, Yanma, Gilgar, Sneasel and Piloswine will have evolutions in Gen IV, so we do need to stock their candies, instead of using all of them for their power-ups.  The same post also mention Porygon2 will have its evolution in Gen IV, so Porygon candies are still needed.

See post here:  Pokemons for Gen 4
Evolution forms of Magneton, Lickitung, Rhydon, Tangela, Electabuzz,
Magmar, Porygon, Togepi, Aipom, Yanma,
Murkrow, Misdreavus, Gilgar, Sneasel, Swinub

Even more forward thinking to future generations, see thread here and reference here.

Region Exclusive

Tauros nearly disappears nowadays.  Farfetch'd is not bad, still availabe in Hongkong.
Screenshot from https://pokemon.appx.hk/ for Farfetch'd and Corosal.

The update region-exclusive map is here: (embedden as well)





Saturday, February 18, 2017

Generation II Early Data Analysis

Last Updated: 2/20/17 8:30am.

Availability Analysis: Wild Pokemons not yet been reported in San Diego:
  • Unown, Corsola.  (Note: Mareep, Pineco and Phanpy was mistakenly filtered out by all trackers.  Fixed in some now).  Corsola now seems to be tropical region exclusive.  Neither Unown nor Entei has been deteced by any trackers.  Unown does exist in wild, apparently all trackers have trouble detecting it.
  • Also not reported:
  • Probably not in game: Delibird, Smeargle, Celebi, Ho-Oh, Lugia, Suicune, Entei, Raikou
  • Babies are not in wild: Cleffa, Igglybuff, Pichu, Togepi, Magby, Elekid, SmoochumTyrogue
  • Regional: Heracross
  • Gen2s that require items in evolution are not available in wild, this incluidng those from Gen1 including Porygon2, Bellossom, Slowking, Steelix, Kingdra, Scizor, Politoed, as well as Sunflora from Gen2.  Despite people have shown screenshot, I have not read evidence that these were caught in wild (unless there is a bug in multiple trackers). Blissey and Crobat are availabe in wild, because their evolutions do not require stone. We have not seen Umbreon and Espeon in wild in San Diego.  We have data for 10523 rare spawns and these were not observed. So I can say for sure this is the case!
Coronado Spawn Rareness Analysis

Coronado (including Strand) spawns during 2/18/17 2:30am-2/20 7:00:pm (52 hours), Inland data are collected from UTC, Rancho Penasquitos and Zip code 92116.

Spawns in San Diego:
Evolution Level 1. Low counts for Mareep and Phanpy were due to tracker filtering.

Evolution Level 2-3. Notice Girafarig, Ampharos and Forrestress are more rare than Feraligatr, Jumpluff and Meganium. (Do trackers filter them?)

Rareness for Coronado Only:
Evolution Level 1

Evolution Level 2-3

Spawn Location Preference Analysis: Coronado versus Inland
Note: Coronado data includes some common ones but high IV, e.g., Hoothoot, Houndour, ignore those.
Comparison between Coronado Data vs Inland Data


Thursday, February 16, 2017

My Game of Pokemon GO: Stage ? - Region Exclusives

This is the last blog entry of my Pokemon GO Gen I series.  Gen II is released today with new sets of silhouette, which is why I am rushing to backup my Gen I memory data onto this disk space.

Tauros is exclusive to North America, Farfetch'd to Asia, Kangaskhan to Australia/New Zealand, and Mr Mime to Europe.  Except Tauros, the rest three certainly are out of reach (can no longer be hatched from eggs either).  The best I could do was to meet them in gyms, which was mistakenly counted as a failed encounter in the game.  Nevertheless seeing a Pokemon in a gym adds a clickable silhouette into your Pokedex.  Collecting all three silhouettes could be a practical relaxed definition for completing of the global Pokedex.

My collection of the regional silhouettes.  Mr Mime was seen in a gym near my house;  Farfetch'd in San Diego Airport on my trip back from Cancun;  Kangaskhan at Coronado Island.

Resources Saved for Future Intercontinental Trips

This section is just for my own bookmarking purpose.

Region-exclusive nests: https://thesilphroad.com/atlas, however, not well maintained.

Kangaskhan in Sydney:
However, Kangaskhan was declared to be as common as Pidgey and were filtered out from the map.  The explanation for this was humorous
Q: Hello, this is Pidgey. Why am I not on the map? I demand a refund #PidgeyLivesMatter.A: Hello Pidgey, THERE ARE SO MANY OF YOU, so to keep your trainers sane, we filter you out. Bellsprout, Caterpie, Clefairy, Ekans, Golbat, Goldeen, Horsea, Kangaskhan, Krabby, Nidoran♀, Nidoran♂, Oddish, Paras, Pidgeotto, Pinsir, Poliwag, Psyduck, Rattata, Sandshrew, Slowpoke, Spearow, Staryu, Venonat, Weedle and Zubat are also not on the map but they didn't complain, did they? #AllMonsAreEqualButSomeMonsAreMoreEqualThanOthers.


Farfetch'd:
It was mentioned that Farfetch'd spawn frequently nearer Taipei 101.

Mr Mime:

My Game of Pokemon GO: Stage VI.3 Lapras Map in San Diego

This blog shows statistics based on 94 recent Lapras Twitter reports collected from public feeds, not including private trackers.

80% of the spawns are in Coronado Island (blue), 16% in Oceanside (Carlsbad) (orange) and with only 4% in La Jolla (La Jolla Shores, La Jolla Cover and Bird Rock) (red). Lapras does occasionally spawn in Marina based on what I have seen in private trackers, but detailed data is not available.  So Coronado is definitely the place to go.  On a lucky day, we could see up to six spawns on the island (top bar graph).


We always wonder whether Lapras is more likely to appear at night or during weekends, or in a raining day (rare in San Diego though).  There appears to be a dip on Saturday (bottom bar graph), however, that is only on the edge of statistical significance (p-value ~ 0.05).  Lapras seem to spawn a bit more frequent at night time, with 57% between 8pm-8am and 43% between 8am-8pm.  In general Lapras spawns pretty uniformly every day throughout a week, it might take a short lunch and midnight break (middle bar graph).  However, catching a Lapras is just like fishing, patience is your best token.



My Game of Pokemon GO: Stage VI.2 Lapras - Programming Exercise

The game is now pretty much over for me, except a few lose ends to tighten up.  I walked enough Togepi candies but decided to hold on to the evolution, as Silph Road users suspected it might be available in the wild (yes, this is confirmed today with Gen II release!).  I walked enough Porygon candies ready for generation-2 evolution (but Valentine Event later just gave away Porygons).  I started walking Lapras to gain candies for power up.  My two friends were still one Pokemon away from their completion: one needed a Porygon and another needed a Lapras.  Time for me to get my hands wet and put my programming skills into use.

By this time, I had been monitoring Silph Road and were aware of a new wave of Twitter accounts was booming.  Coronado Island is just too far to patronize often from our locations, based on Twitter feeds I discovered Lapras occasionally spawned in La Jolla and Oceanside as well, which are closer to where we live.  However, Twitter feeds contains tons of Pokemons we did not care, i.e., 99% of the feeds were junks to us, making it unuseable.

One Friday night, I wrote an application, streaming feeds from several local Pokemon twitter accounts, filtering for keywords Porygon and Lapras, once located, it sent a text message to my friends.  This quickly led to successful Porygon captures.  One text message a few days later brought a nearly level-28 Lapras at La Jolla to my other friend.  All members of our Pokemon team now had completed the Pokedex! Hurray!!

The program streams messages from multiple Twitter accounts, filtering out those not containing keywords Porygon, Snorlax or Lapras in the text body.  It sends Google Map location and time-to-hide as text messages. 

My Game of Pokemon GO - Stage VI.1: Lapras, I See You!

Lapras is a myth.  It only patronizes certain hot spots along the coast, with zero interest in showing up at other random locations.  This behavior is drastically different from Snorlax or Dragonite, I would say with confidence that Lapras is the most desirable Pokemon by most players.

A Lapras artwork by our youngest Pokemon GO team member SZ.  Lapras is loved.

After the FastPokeMap setback, developers realized a 100%-open-source community-transparent hacking model simply would not work, some decided to pursue an underground model.  Cracked hash keys were kept as a secret and were only accessible to very few developers.  Once a new hash key was cracked, hackers provided paid hash-key encoding services, so that subscription apps such as PokeAlert on Android, PokeWhere on iOS could continue to call Niantic API.  A paid subscription model was established; cash incentive enabled developers to continue their hash key cracking efforts version after version.  Even after Niantic introduced reCAPTCHA to block bot accounts, developers figured out how to rely on crowd sourcing efforts to crack reCAPTCHA (or used paid human solvers).  Niantic then stopped providing time-to-hide (TTH) information in their API (TTH data is not returned until the last 1.5 minutes of a spawn), i.e., you could no longer know how many minutes left for a remote spawn.  Developers figured out a clever workaround.  For example, if you continuously monitor a spawn point, the program would soon figure out exact when a Pokemon is expected to be spawned at that coordinate and how long that spawn lasts (repeative scans detect when the Pokemon disappeared).  As each spawn occurs at the exact minute at each hour, computer program will know not only TTH, but also when to scan a spawn spot.  How clever!  No matter what Niantic came up, developers always found a counter measure.  However, Pokemon map was still not good enough for catching Lapras, it was simply too rare.

As tracker packages resurrected and became more mature and easier to setup, a volunteer setup a private feed for Coronado using a Slack server.  I signed up without hesitation.  It was also discovered that Niantic had increased the spawn duration for rares from 15min to 30min in December, probably because the change of their nearby feature required players to travel further to catch a rare.  Thirty minute made it feasible for me to go to the island by myself.  Learned from our previous mistake, my friend and I decided to visit the island the day before Christmas to hunt for Lapras, hopefully before the history repeated and Slack server would be shut down by Niantic.  We got lots of Grimers, Dratinis and a Snorlax on that day between 8am and 3pm, however, Lapras did not show up.  We were nevertheless confident that the Slack feed worked and agreed to come back after the new year.

On the New Year’s day, the morning after I was back from Cancun, I drove to the island at 10am alone.  Every chime on the phone brought up a hope.  Finally one chime at 10:45am carried the long-overdue alerts for a Lapras near Hotel de Coronado with plentiful 26m left and I was merely 5min away.  Still excited, drove, parked, ran to the beach, the moment I have been waiting for three months to complete my Pokedex was finally arrived!  I had tons of Ultra balls and Razz Berries stocked, preparing for the toughest final battle I could imagine.  Well, the difficulty of catching Lapras is nowhere near the challenge of catching a Dragonite, one berry and one Ultra ball, game over!  Mission accomplished!!  I made it!!!

My very first Lapras was caught near Hotel De Coronado on the New Year's Day. Another gorgeous one was caught in a week after at the Dog Beach (3rd screenshot).



My Game of Pokemon GO - Stage V: Babies

Egg hatching was not totally fruitless. In the mid December, seven baby Pokemons were introduced into the game; they could not be caught in wild, but could only be hatched.  Based on my rich experience in hatching ($$$, time and distance walked in combine!), I knew those rare 5km babies were just as unattainable as Porygons, therefore, I was not going to waste more money on babies.

Then came the Christmas event, where the probability of baby-containing eggs were significantly boosted by Niantic.  I was taking a vacation in Cancun, but I knew this was the only window to get babies without spending a fortune.  With the unknown but significantly-increased probability of getting baby eggs, the return-of-investment in incubator-purchasing was more justified.  Besides, I always had a theory that when the game was first released, the probability of hatching Lapras was way higher based on anecdote stories I heard, but obviously that theory cannot be proved right or wrong, anyway, I did not want to miss this special baby-introduction phase.  It was fortunate that I could walked in the warm sea breeze in the middle of December, which also helped to keep my body in shape, while we were being fed like pigs in an all-inclusive hotel.  With tons of luck, all babies were hatched and I even got two Togepei, one could be used for evolution later.  I have not bought a single incubator since, hatching is generally a terrible strategy for players living in San Diego -- literally a Pokemon GO paradise.
100% loaded to maximize the baby-hatching probability during the Christmas event.  The orange incubator is the free one provided by the game, all other eight costs $1.5 each and were good for three hatches only.

Before the Christmas event, I hatched 346 eggs after walking 502km (recorded on Dec 16, 2016), by now, I have walked 760km and hatched 489 eggs.

Eggs hatched and distance walked so far.

I got a total of 41 10km eggs, including enhanced 10km egg probability during the Christmas baby event, therefore my 10km egg-rate is a bit biased and is misleading above 7.4%.  I believe 7.4% is a solid estimation for the probability of obtaining a 10km [see previous post].  Here is the hatch chart of all my 41 Pokemons.  You will not find Lapras there.


Birth Certificates

Each Pokemon has a birth certificate recording when it was caught/hatched and where it was caught/laid (if from an egg).  I have Pokemons collected in various locations and it is fun to check these certificates to bring back some good memories.

Example birth certificates: Togepi is a Mexican baby born on 12/28/2016.  Visa was not required to bring babies from Mexico into the US, confirmed (well, at least before the new president).  Smoochum is a Houston baby born on 12/26/2016 (egg obtained at airport).  Dratini is actually not a Mexican baby. Pokemons caught at south end of the Coronado Island were mistakenly marked as Mexico.  The map used by the game is not accurate, the Dratini is in fact a legal US citizen.

My Game of Pokemon GO - Stage IV.2: Hatching

Although FastPokeMap attempted a few short-lasting recoveries, soon after developers decoded the hash key, Nantic patched and used a new key.  FastPokeMap was deemed to be a loser in this cat-and-mouse game and Twitter accounts were never restored.  With catching Lapras and Porygon in the wild becoming hopeless, the only route remaining was egg hatching.

Porygon could be hatched from 5km eggs and Lapras from 10km eggs.  Despite different statistics exist, none was collected at large enough sample size, it was the best to assume all Pokestops provided equal chances of 5km and 10km eggs and all Pokemon types were hatched with equal likelihood.  There were about 41 species in 5km eggs and 15 in 10km eggs.  The number of hatches requires for a 50% chance of obtaining a Porygon and a Lapras could be calculated using the simple equations below:

Porygon: $1-\left(\frac{40}{41}\right)^n=0.5$,
Lapras: $1-\left(\frac{14}{15}\right)^n=0.5$,

where $n$ is the number of hatches required.

The probability of hatching at least one target Pokemon versus number of hatches for the corresponding egg types are charted below.  For a Porygon (orange 5km curve), we need 28 5km hatches to archive 50% and 65 5km hatches to archive 80% chance.  Lapras (purple 10km curve) requires 10 10km eggs for 50% and 23 10km eggs for 80%.  Not bad.  However, when we get an egg from a Pokestop, ~80% chance it is 5km, but 10km eggs are rare.  The most reliable statistics probably comes from a recent Pokemon fan KSBS in Singapore [1], who kept a record for 3175 egg hatches!  He got 236 10km eggs, giving a probability estimation of 7.4%.  So 80% chance for Lapras requires 23/7.4% = 311 eggs.


In hindsight, it is most likely a wrong assumption that all 10km species are equally likely.  KSBS' data suggests Lapras has a much lower chance: 8/236 = 3$\pm$1%, compared to an average rate of 1/15=6.7%!  So the more realistic Lapras hatch curve is the blue curve; one needs 47 10km eggs for one Lapras, which translates into 47/7.4% = 635 egg hatches!  This estimate matches my own experience and what I read recently on Silph Road.

Either 311 or 635, with an average 5km-walk per egg hatch, you would have to walk 1500-4500km for a Lapras.  It was pretty obvious to me that this was time to pour cash into the game -- one had to buy incubators.  The game came with only one infinite-use incubator, I needed to buy eight 3-use incubators at $1.50 each, then I could reduce my walk by nine fold using parallel processing.  Cash was injected, no sight of Porygon and Lapras.  I passed the 50% hatch point, then the 75% line, still trying ...  Eventually I got two Porygons, both with IV above 85%.  

By Dec 16, I have walked over 500km and hatched 346 eggs and I decided to quit.  With $0.50/hatch, you can figure out how much I spent in this hatching stage without accomplishing the goal.  One common mistake people tend to make, including myself, is the intuition that our past hatch failure increases the chance of future success, i.e., the more failed hatches, the next one is closer to be Lapras.  However, textbook tells us a different story, after each failure, I still need another 311 or 635 eggs in order to have 80% chance of gaining Lapras in my next attempt.  The history of 346 eggs has no effect on the future and all those incubators were simply a waste without leaving a footprint.  Getting Lapras this way is a lottery most of us cannot win.

Chasing Snorlax

Catching a rare without trackers was nearly mission impossible, but I did manage to catch a Snorlax one night 100% on my own effort.  As I walked entering the "Snorlax Detected" point from the north, the big fat fellow appeared in my sightings, which means it was 120m away.  I started running zigzag as the green arrows indicated on the map.  The yellow dash line outlined the approximate 120m boundary where Snorlax disappeared from my sightings, which made me think either Snorlax has despawned or I have moved too far, fortunately it was always the latter.  After an intense 10-min blanket scanning, a colossal Snorlax, weighted 693.05kg, entered my field of Poke vision!  Postmortem analysis certainly suggested a much better alternative tracking strategy exist, but I was not going to repeat that again, it was simply too tiresome.
Snorlax caught by brutal-force blanket scanning on Nov 11, 2016.


Reference
  1. https://www.reddit.com/r/TheSilphRoad/comments/5rsyqn/coming_out_lvl_38_3175_eggs_hatched/

Wednesday, February 15, 2017

My Game of Pokemon GO - Stage IV.1: No More Trackers

Three candies were earned when a Pokemon was caught, one extra candy for transferring it, so four candies can be obtained per Grimer caught in wild.  To get a Muk, I would need to catch ten more Grimers, which was nearly mission impossible.  I was so close to complete the Pokedex at this point, I was determined.  Hope came when Niantic introduced the "buddy" feature into the game. I could then choose Grimer as my buddy, one candy earned per 3km walk.  Forty candies would take 120km!  Tough but at least it was a path forward.  By the time I walked about 80km and earn 27 candies, Niantic launched the Halloween event trying to attract players back to the game.  During that special week, walking distance was reduced to one Grimer candy per 0.75km, a sweet deal and the remaining candies were piece of cake.  My Grimer was finally ready to morph into a Muk!
Walking my Grimer buddy for over 81.8km. 


I was still missing the last three: Porygon, Snorlax and Lapras.  Tools had sharpened by that time, Twitter accounts started to report rare Pokemons.  We would just needed to be sitting at the beautiful beach of Coronado Island.  When Twitter feeds reported the precise location of Lapras, drove to it.  However, with only maximum 15min for a given spawn, I figured it was not very productive to do the hunting on my own, as I would barely have enough time after I drove across the island and after managed to win a parking spot from other hunters.  My friend and I decided to visit the island in two weeks, which would be mid-October, as we both had family business before that.  This decision turned out to be a huge mistake, one that I had to pay nearly $200 for.

A pleasant surprise came on a Sunday morning.  I happened to open up FastPokeMap and pure luck stroke me.  I saw a Snorlax one block away from my house with 4min left.  Still could not believe what I saw on the screen, I hurried out in my pajamas!  One down!  Porygon and Lapras, here I come – in just two weeks!  But things turned south quickly.

FastPokeMap got millions of usage per day.  Its database grew to the scale that requires NoSQL database to handle, the web traffic also required the latest Big Data Cloud infrastructure to support.  Spawn rules had been discovered through data mining, biomes patterns were explored.  The FastPokeMap movement certainly caught Niantic’ attention.  Bad news came in the morning of October 8th, 2016.  Niantic introduced more stringent encryption mechanism, where all existing programming interface became useless.  FastPokeMap was shutdown, all Twitter feeds sitting on top of these databases or relying on old interface were dysfunct and were never recovered.  Our plan for the final battle at Coronado Island was complete scraped, as it was pointless to roam on the island without Twitter feeds.  This was the start for a Pokemon dark age.

I wrote the following in that dark morning (hint: my friends were working on drug discovery, so I used related jargons):

"Pokemon hunting is like lead hunting.  In the early days, we could only use "nearby" feature to explore our neighborhood, which was like we shaked test tubes to read out the activities of each individual compounds.  Later Go Radar app provided us a 96-well little gadget, then came FaskPokeMap, which was a modern 1536-well ultra-high-throughput (uHTS) screening system, the rule of PoGo game was reshaped forever.  Several refinements occured afterwards, such as the use of Focus library.  Hunting Bulbasaur at the Hour Glass park was just like finding Gleevec from a Kinase library.  Use libraries of Known Drugs with high hit rates was like visiting hot spots, such as Del Mar, La Jolla and Coronado Island.  Not to mention Twitter feed, that was basically monitoring top journals such as Nature, Science and Cell.  Once a target was published in these journals, we race to win!  Now uHTS has gone, Nature Science Cell has been off the shelf, where is our next innovation?  Beginner players may still rely on Focus Libraries or Known-Drug Libraries, pleasant surprises could still occur once in a once.  However, for our advanced venture capitalists, where is our next Blockbuster opportunity?  Our technical department (hackers) is still fighting hard to crack the code and repair the tools we used to have, but this also presents a opportunity for brand-new ideas.  One possibility is to embrace open science and rely on crowd sourcing, such as use a human-driven GO Radar system.  Let us imagine a few thousands of us standing on the Coronado Island forming an array at 100-meter interval.  Each person is in charge of monitoring his 40-meter sightings radius.  Once a rare Pokemon is discovered, we tweet and disseminate the information immediately.  Forming a coalition is the key to the success of this open science model.  Since we have tasted the power of those modern technologies, we can no longer go back to pick up test tubes and hope to find a Lapras, a Porygon, or a Snorlax that way, nor should we unrealistically dream about obtaining a magic egg, which then hatches into what we desprately need.  These are not real options.  Let us take a break, rejuvenent, wait for the next technology revolution, maybe we will meet again in a total different PoGo battle field."

The original text was sent to my dear Pokemon friends in Chinese and I intended to keep that copy here:

"Pokémon hunting就如lead hunting。我们最早只有在neighborhood用nearby,有如手动地晃动试管读取每一个compound的信息。后来Go Radar给了我们96-well的小机器,FastPokeMap则是1536-well的uHTS,从此改写了游戏的规则。然后又出现了几种技术革新,比如用Focus library。去Hour Glass抓Bulbasaur就是在Kinase Library里找Gleevec。用Known-Drug Set犹如去德拉梅,拉赫亚,科岛这些Hit Rate极高的Library。当然还有follow Twitter feed,那就是跟踪Nature,Science,Cell,一旦别人有了发现,就比谁跑的快!现在HTS当了,NCS杂志下架了,下一个技术革命在哪呢?小盆友们当然还可以用Focus Library,Known-Drug Set,还可以找到惊喜,我们这些只看中Blockbuster的风投该何去何从?技术部门绞尽脑汁也许能修复那些曾有的工具。但这也是给新技术诞生提供了摇篮。一种猜想是embrace open science,用Crowd Sourcing,比如协同合作人肉狗雷达。想象在科岛,几千个我们站成间距100米的阵列,每人负责监视和第一时间Tweet半径内的发现。联盟是open science成功的关键。由奢入简,重新拾起试管去筛找Lapras, Porygon, Snorlax,或者幻想捡到一个magic power egg都不是选项。只有养精蓄锐,拭目以待,迎接下一个场革命,说不准我们会在一个新的战场上。"

My Game of Pokemon GO - Stage III: Nest Hunting

Although the program PokeVision was gone, developers realized Pokemon’s API could be used to create new generation Pokemon maps.  Among all the map programs, the best of the best is called FastPokeMap [see previous blog]. FastPokeMap allows one to click on a map and pretend to be anywhere, this way I could mouse click and mapped out all Pokemons even in places, where no player was physically on ground zero.  I started monitoring two shopping malls and other neighborhood hot spots within 8-min driving distance and my desktop was my "Map Room".  As Pokemon spawns only lasted 15 min at that time, I needed to response to spawns just like fireman would respond to 911 calls.

At this stage my play was extremely purposeful, as I had already hunted most not so rare ones in beach locations.  My only remaining goal was to complete the North American Pokedex, nothing else.  So I did not battle gyms, I did not hatch eggs, I only caught enough Pokemon to gain level twenty in order to get Ultra balls, then pretty much only caught the Pokemons I needed, i.e., either it had to be a new species, or the ones I could exchange for evolution candies to eventually gain new species.  This was why my trainer level is still far behind my friends till this day, as I was not actively gaining XP points (I finally reached level 30 before my seven-month mark onFeb 12, 2017, still behind my friends) .

As my only goal was to collect new Pokemons, I did not battle for the first four months. My first Gym standing was only till Nov 25, 2016 at Costco.

There were certainly pleasant surprises during this period.  I hatched a Tengela from a 5km egg the moment I got home one afternoon.  I discovered three hot spots within 100 meters away from home that spawned rare Pokemons at exactly the same minute every hour, which was how I got my second Grimer.  One night I saw a humongous silhouette in the sightings, while driving back from Del Mar beach.  I pulled up to the side and used FastPokeMap to locate my first Dragonite!
Dragonite (center) encountered by super good luck on Sep 16, 2016.


Rare but evolvable Pokemons can be secured through hard work, such as Gyarados could be evolved from the common Magikarps.  Even though it requires hunting 100 Magikarps, very time consuming but nevertheless manageable!  However, Venusaur, Charizard, Vileplume, Victreebel, Gengar and Muk were the toughest, as their ancestor forms were already rare.  At that time I already started following Silph Road on Reddit and some local Pokemon Facebook groups.  Through crowd sourcing, I learnt there were something called "nests", where rare Pokemons spawned at the rate of one per 10-20 minutes or so, much higher than background occurrence.  Nest hunting was probably one of the most memorable game period.  I visited many small parks, places I would never visit otherwise.  Gastly were harvested at the Del Mar Fairground Parking lot at night; Bulbasaur at Eucalyptus Park in Chula Vista; Bellsprout at Hilltop Community Park in Rancho Penasquitos; Charmander at Martin Luther King Jr Memorial Park in Oceanside, and Oddish at the beautiful Liberty Station NTC Park.

Screenshots taken at the peak of spawn in nests.  Each hunt lasted about two hours.

I got three Grimers with ten candies, I still needed fifty candies to produce a Muk.  Grimers were super rare and nearly only sporadically spawned in Coronado Island.  That was a problem.

Bonus Material

Nests migrate, i.e., every few weeks, a Jynx nest may turn into Jigglypuff nest, then into a Nidorino nest, etc.  The trainer community start their crowd source efforts and maintain a Nest Atlas [1]. However, the atlas is not well maintained, probably because beginners do not know about nests and advanced trainers do not care about nests.  Really rares do not have a nest.

Reference
  1. https://thesilphroad.com/atlas


My Game of Pokemon GO - Stage II: Location, Location, Location

One day my colleague at work mentioned how La Jolla Cove was full of PokeStops raining Sakura petals at night, I realized we did not have to go to Canada to get a Horsea after all.  How hard could it be to access beach, which was only three miles away?!  Fun started at Del Mar Powerhouse Park near 14th street - Seel, Magnemite, Slowpoke, Poliwag, Shellder, Voltorb, even rarer ones such as Gastly, Cloyster and Haunter surprised me from time to time.  Some after-dinner trips to La Jolla Cove added Seaking, Onix, Tentacruel and Aerodactyl.

Then I read Niantic shut down a tracker program called PokeVision.  Holly cow! I did not know there had been a tracker, until its death toll made the headline.  PokeVision was a real-time Pokemon map, which offered its users a “Pokemon GPS”!  Too late for the PokeVision party, lesson learned, I started monitor Pokemon online communities.  Homework is unavoidable if you want to excel in this game.

My research soon pointed to an iPhone app called GO Radar.  Players using rooted Android devices were collecting sightings from their Pokemon GO apps and sharing the data on the GO Radar server.  GO Radar app allowed me to see precisely where nearby Pokemons were, how much time was left for the catch, furthermore, it allowed me to see where rare Pokemons were clustered in San Diego – sweetest spots in San Diego include downtown, Coronado Island and Oceanside Pier.  I could clearly see what later known as nests, e.g., Seaport Village was a Drowzee nest, and Coronado Ferry Landing was a Voltorb and Magnemite nest at the time.

I finally took my family to Coronado Island, a sacred pogo playground known as “the quad”, where maybe a hundred players camped on the lawn in the center of the Orange Avenue surrounded by four Pokestops lighted with lures all night long [1].  I remember that night as the night I hunted our first Grimer, however, history remembers that night as August 20, 2016, the night Chinese Women’s Volleyball team won its hard-earned gold medal in Rio de Janeiro.  In another lucky Sunday morning, I got a Blastoise, a Hypno and several Jynx (a nest) under the Coronado Bridge within an hour.  My Pokedex exploded in the second month.  Location, Location, Location!  This is what this game is mostly about!

The "quad" at Coronado (August 20, 2016)

This is why it is known as the "quad".

With GO Radar, I caught my first Grimer that night.

What was the most exciting moments in the game? Whenever I saw a silhouette pop up in the sightings, indicating a new species was nearby.  My adrenaline shot to the roof, when that happened. This screen shot was taken in the lucky Sunday morning under the Coronado bridge.  The Blastoise, embossed in red, was then located by FastPokeMap.

Technical Excursion – Early Days of Pokemon Tracker

Google map divides the surface of the earth into cells using a library called S2 [2].  Cells have 30 levels, where the most granular level 30 represents an area as small as 0.75 cm2!  Pokemon GO uses level 15 cells, which corresponds to a parallelogram of roughly 200 meters in dimension.  The parallelogram ABCD in the screenshot below, taken during my Gastly hunt at the Del Mar Fairground Parking lot, shows what an example level-15 cell looks like.  The reason the cell shape is not a square is because the spherical earth surface is projected onto six squares of a cube surface, which caused skewness for locations other than equator.  Parallelograms are tiled all over the map, ABCD is just one example.

Now imagine a player stands at the pin O, all Pokemons spawned within 40 meters (blue circle) can be spotted on the app screen and are catchable.  Pokemons within 120 meters (red shaded circle) showed up as nearby (see the bar at the very top of the screenshot with the first Gastly circled in purple) without knowing their exact locations.  The player is now supposed to move around and hunt for the first Gastly within the red circle, i.e., you drag the blue circle using your body to uncover a hidden target.  As a nearby Pokemon could be anywhere within the red circle and each Pokemon spawn only lasted 15min, the success rate of locating a nearby Pokemon was not impressive.

Early versions of trackers took advantage of the Cell identifier associated with a nearby Pokemon (Cell ID data was provided via programming interface) and let us know within which parallelogram Cell we should be hunting, this significantly reduced the guess work, as we now know the monster is within the purple embossed area (common area between ABCD and the red circle)!  Although several click-to-scan operations were still required within this region to move the blue circles until the target was located, it was already a giant step forward.  Later versions of trackers scanned a number of blue circles simultaneously on behalf of the player, therefore was able to expose every single Pokemon within the red circle under one click!  This led to the enormously popular site call FastPokeMap.  Before FastPokeMap, some players had to use GPS-spoofing devices to fake their locations into Pokemon-dense regions, such as the Central Park at New York City, some even virtually travel to other continents to catch regional-exclusive rare ones.  Ninatic gradually tightened up the holes and banned such accounts.  During that campaign, Niantic once made a mistake and ban a whole country from playing Pokemon GO for one full day [3].

FastPokeMap was revolutionary as it created more than a million bot trainer accounts and used them to scan on behalf of players at its server side, therefore, it was 100% safe for players to use FastPokeMap with no association to their real accounts.  FastPokeMap was also a crowd-sourcing service, its scan results triggered by one user were cached on the server for all users to view.  Popular spots that had many players naturally had denser Pokemon Map and therefore attracted even more players.  The FastPokeMap screenshot below shows how insanely busy Manhattan was.  By late September, FastPokeMap has reached 10 million daily page views and 3 million unique users per day [4], thanking to all the big data and cloud infrastructure used to support such a massive operation.
Manhattan under FastPokeMap!


Reference
  1. http://www.sandiegoreader.com/news/2016/aug/25/stringers-pokemon-goes-coronado-and-stays-there/#
  2. http://blog.christianperone.com/2015/08/googles-s2-geometry-on-the-sphere-cells-and-hilbert-curve/
  3. http://www.geek.com/tech/one-pokemon-go-player-got-all-of-belgium-banned-1667630/
  4. https://pokemongohub.net/fastpokemap-reaches-10-million-daily-views