Instagram's war on me continues apace

Dear Lazyweb,

Can anyone give me working instructions on how to mitmproxy the Instagram app, from macOS, using either a real iPhone, or an emulated Android or iOS device?

When I proxy the iOS version, Safari and Twitter work, but Instagram says "Couldn't refresh feed" or sometimes "no Internet connection". It's detecting the proxy somehow. Maybe cert pinning?

It appears that there are people out in the world successfully proxying it on Android, but I can't figure out how to proxy the Instagram app in an Android emulator. There are instructions to make proxying work with non-Chrome apps, and those involve doing "adb root", but that doesn't work on Android installs that have Google Play on them, and without Google Play, how do I get the Instagram app installed in the emulator at all? (Note, I know nearly nothing about Android.)

Previously, previously.

Tags: , , , , , , ,

28 Responses:

  1. Wilhelm Fitzpatrick says:

    You can usually find extracted .apk's for most apps on the interwebs. APK Mirror is a good choice for being reasonably confident it is the real thing and not pumped full of Chinese malware:

    https://www.apkmirror.com/apk/instagram/

    Once you have the apk, you can use "adb install" to push it to the emulator.

    You might need to set up an ARM image in your emulator (instead of x86) since the extracted images are for ARM ABIs. It might also not matter if the app doesn't have any native libs (I don't know for Instragram)

  2. Nick says:

    How do nightclubs not owned by former netscape engineers get anything done

    • Thomas Lord says:

      They hand over the business to corporate booking agencies, pick their vibe and offerings off the retail rack, and generally give up on being more interesting than, say, a combination Pizza Hut and Taco Bell?

      Just guessing....

      • John says:

        See if you can download the .apk for instagram on APKMirror and replace the CACert in the APK with your mitm one.
        Use APK Studio to open the .apk itself, then look around for any CA cert. If you're lucky, you'll easily be able to delete the old CA, then put in yours.

        • John says:

          If you click 'Reply' on a post, refresh the page, then put your comment in the now non-reply field, it still posts the comment as a reply to someone.

          • Thomas Lord says:

            I took "delete the old CA, then put in yours" as a poetic description of the war on fun that hurts nightclubs, so it works out OK anyway.

  3. jwz says:

    I think I'm almost there, but it's not working. I did this:

    1: Android Studio, upgraded everything.
    2: Created new emulator, Nexus 5 API 28, "Google API" not "Google Play"
    3: cd ~/.mitmproxy/
    4: openssl x509 -inform PEM -subject_hash_old -in mitmproxy-ca-cert.cer | head -1
    5: cp mitmproxy-ca-cert.cer c8750f0d.0
    6: emulator -avd Nexus_5_28 -writable-system
    7: adb root
    8: adb shell "mount -o rw,remount /"
    9: adb reboot
    10: adb push c8750f0d.0 /system/etc/security/cacerts
    11: adb shell "chmod 664 /system/etc/security/cacerts/c8750f0d.0"
    12: Verify that it is a system-trusted cert by looking at Settings / Security & location / Encryption & credentials / Trusted credentials.
    13: Preferences / Wifi / "AndroidWifi" / pencil-icon / Advanced options: Proxy: Manual 10.0.1.2 8080
    14: Open a page in Chrome: it goes through the proxy. Yay.

    15: adb install -r com.instagram.android_123.0.0.21.114-188791703_minAPI23\(x86_64\)\(nodpi\)_apkmirror.com.apk
    16: adb install -r com.twitter.android_8.24.1-release.01-18241001_minAPI21\(arm64-v8a,armeabi-v7a,x86,x86_64\)\(nodpi\)_apkmirror.com.apk

    But Twitter can't log in, doesn't proxy; Instagram can't log in, doesn't proxy.

    • angled says:

      Can you try CharlesProxy instead of mitmproxy to see if proxying works at all?

      https://www.albertomoral.com/blog/get-data-from-instagrams-private-api

      https://www.charlesproxy.com/download/

      • jwz says:

        There's like no chance that mitmproxy wouldn't at least log a connect(). Also Chrome works.

        • angled says:

          I ran Instagram through CharlesProxy on iOS 13, and it seems like Instagram isn't trusting the installed certificate, here's a screenshot, and I've pasted the full errors below. I installed the Charles certificate per their guide, https://www.charlesproxy.com/documentation/using-charles/ssl-certificates/

          <img src="https://i.imgur.com/gQBUk4I.png&quot; /

          Failure

          SSL handshake with client failed - The client supported protocol versions [TLSv1.3, (D)TLS--5.26] are not accepted by server preferences [TLS12]

          Notes

          You may need to configure your browser or application to trust the Charles Root Certificate. See SSL Proxying in the Help menu.

          TLS Alert Code

          protocol_version (70) - The TLS protocol version proposed by the client is not supported by the server

          so have fun...

          • Nick Lamb says:

            Cute. Yeah, so they refuse to talk TLS 1.2 or earlier, knowing that their servers all can do TLS 1.3 so if you can't then you must be an impostor.

            This is extra confusing for a naive TLS 1.2 server because TLS 1.3 clients don't say TLS 1.3 where it expects. Middleboxes freak out if you say you speak any version they don't know about even though that's the whole point of version negotiation, so an actual TLS 1.3 client says "Hi, I'm resuming a TLS 1.2 connection which you'll remember from earlier, also I support a cool new optional extension FlyCasualThisIsReallyTLSv1.3 with parameters that look exactly like ECDHE key exchange" and then a TLS 1.3 server goes "Right you are, TLS 1.2 resumption, I too know FlyCasualThisIsReallyTLSv1.3 with parameters that look like my half of a key exchange" and then like in Portal 2, "Surprise! We're Doing It Now" everything after that is encrypted and the middlebox figures this must be an old session it has previously allowed through so no need to intervene. If you as server don't understand TLS 1.3 you go "Um, what? No we've never spoken before, maybe I was rebooted. Anyway TLS 1.2 is fine but let's start from the beginning as usual and I don't understand that fancy new extension" and that incurs the round trip cost we're used to for TLS setup.

            The "real" clients are definitely key pinning. So you will need to obtain "cracked" clients which patch out the pinning checks or they'll abort even if you are able to MITM the connection.

            • jwz says:

              Ok but I'm able to proxy the iOS Twitter client, which means it's not pinning. It would be weird for it to be pinning on Android but not iOS, wouldn't it?

              I've yet to see any proxy hits from an app other than Opera, so I don't know that proxying is working with any app. Not sure how to test that.

              • Nick Lamb says:

                It is definitely possible that explicitly hooking up your proxy (in Android Studio?) in step 13 causes apps which don't want to go through a proxy to ignore it completely and just try to connect directly or fail.

                Silently MITMing their networking (so that they can't opt out though they might still detect you and fail later) is a bunch more work for which I have no pre-baked instructions. The least invasive thing that could work for that is to provide lies for DNS answers, claiming your mitmproxy is the real address for any server and much more drastic is to just force all the packets to go to mitmproxy even though they have the wrong destination, then "fix" packets coming back out to show the expected source.

          • jwz says:

            Well, according to this, mitmproxy doesn't support TLSv1.3. And there's this which says "Most of our dependencies, openssl, and their Python libs, should support it though". MacPorts gave me mitmproxy 3.7, but I tried the git head version, 6.0.0.dev, with Python 3.7.6, and Android Twitter and Instagram still don't proxy. I don't know how to tell whether 6.0.0 thinks it supports TLSv1.3 or not.

            But again, proxying iOS Twitter works fine with mitmproxy 3.7.

            Does it seem plausible that the iOS version of the Twitter app works with TLSv1.2 but the Android version requires TLSv1.3?

            Does it seem plausible that the Android version pins certs but the iOS version does not?

            • angled says:

              Maybe?

              For the FB properties:

              https://engineering.fb.com/security/fizz/

              "We have deployed Fizz and TLS 1.3 globally in our mobile apps, Proxygen, our load balancers, our internal services, and even our QUIC library, mvfst. More than 50 percent of our internet traffic is now secured with TLS 1.3. We also deployed zero round-trip resumption (0-RTT) data, the newest feature of TLS 1.3, with our mobile apps."

              which also led to this:

              https://plainsec.org/how-to-bypass-instagram-ssl-pinning-on-android-v78/

              that's a year old and hocusy pocusy.

              for twitter? maybe the android team decided fizz was cool, and the iOS team didn't include using external libraries in their KPIs?

          • jwz says:

            I also tried Fiddler but I can't figure out how to get it's certificate onto the device, iOS or Android. The instructions have me downloading an EXE file, yay. And the magic URL doesn't work.

            • jwz says:

              Ok, I got the cert installed and trusted on iOS, and Fiddler is decoding HTTPS traffic from Safari, but for any apps, it seems to be causing HTTP CONNECT tunnels instead of GET, with no decoding. Their documentation sucks.

        • Nick Lamb says:

          mitmproxy does log connections but only silently to its Event log, if you aren't looking at the Event log you won't see a connection that mitmproxy didn't end up proceeding with.

          • jwz says:

            How do I see this event log?

            I'm seeing it proxy when using Chrome, XScreenSaver, Google Maps, and the Youtube app. Still not working with Twitter or Instagram, though.

            • Nick Lamb says:

              In the command line mitmproxy it was shift-E to see the event log.

              e.g. if I just telnet localhost $MITM_PORT then that shows nothing on the normal mitmproxy UI but in the event log I get a line saying somebody connected, and, when I disconnect the telnet, that they disconnected.

              • jwz says:

                Ok, with both versions of mitmproxy the Android emulator and iOS hardware both show me Events but not Flows from Twitter and Instagram.

                Yes this means that suddenly I'm no longer able to proxy iOS Twitter either, when I definitely was able to a couple days ago. WTF.

                (Also I tried adding SSL_version => 'TLSv13' to my perl code, and that was recognized but didn't help.)

  4. hudson says:

    There is Frida, which allows MITM-like functionality by hooking the SSL library calls in the application instead of trying to proxy the network connection. At hack.lu 2019, Axelle Apvrille showed how she used Frida to trace HTTP/S functions in Android apps (including Instagram around 12:00 in the video) in a way that was much more exact than wireshark MITM of the packet.

    Being able to write python hook for specific library function calls would allow you to drive the application as well, if that is your end goal.

  5. Anonymous Coward says:

    TLS 1.3 support in mitmproxy appears to be implemented in this not-yet-merged pull request: https://github.com/mitmproxy/mitmproxy/pull/3692

    • jwz says:

      When I apply that patch it dies with:
      Requirement.parse('cryptography>=2.8'), {'pyOpenSSL'}
      Hand-hacking the version number doesn't fix it. But also this is confusing because:
      port installed | grep cryptography → py37-cryptography @2.8_0 (active)
      I am in a twisty maze of python virtual environments, all alike.

      • jwz says:

        Edited the version number in setup.py and then it seemed to install 2.8 but then dies with:
        module 'lib' has no attribute 'SSL_CTX_get0_param'

  6. March says:

    This could be caused by a couple of things, jwz.
    Most likely, I'd suggest it's this: https://android-developers.googleblog.com/2016/07/changes-to-trusted-certificate.html
    Since Android 7.0, certain apps can basically ignore the certificates you add to the store - it's really irritating. Try installing Instagram in an Android 6 emulator (Genymotion works).
    They might also have certificate pinning - I've never looked at the instagram app before, but it seems plausible. You could use something like Frida or Xposed to hook any cert-related functions and break the pinning, though.

  7. jer says:

    Try another way.
    Requirement:
    1. Android Emulator with OpenVPN client installed.
    2. Server with OpenVPN and mitmproxy

    Idea: MITM between OpenVPN Server and Internet. Connect OpenVPN client to your server, it will route all traffic from Emulator to OpenVPN server. You can replace OpenVPN with any VPN server that can be proxy.

Leave a Reply to Anonymous Coward Cancel reply

Your email address will not be published. But if you provide a fake email address, I will likely assume that you are a troll, and not publish your comment.

You may use these HTML tags and attributes: <a href="" title=""> <b> <blockquote cite=""> <code> <em> <i> <s> <strike> <strong> <img src="" width="" height="" style=""> <iframe src="" class=""> <video src="" class="" controls="" loop="" muted="" autoplay="" playsinline=""> <div class=""> <blink> <tt> <u>, or *italics*.

  • Previously