Installer is working now. Thank you for your help! Turns out it was a driver issue. I'll describe what worked for me below in case anyone else has a similar problem.
Problem was that I had no driver that could recognize the phone when it was on the recovery screen. Nexus Root Toolkit installs drivers for the home screen and the bootloader screen, but not the recovery screen.
To know that this is the problem, go into command prompt and enter "adb devices". If your phone is connected and on the home screen it should list the Nexus 4 (even if the "name" is unintelligible). If you're on the bootloader screen "adb devices" should list nothing (but "fastboot devices" should list your phone). If you're on the recovery screen, "adb devices" should list your device, but for me it wasn't finding anything while the phone was on the recovery screen.
To fix it using Nexus Root Toolkit, open it and begin the driver installation guide. Follow the instructions until step 3, and instead of the recommended option take option 4 - the custom/manual one. It will guide you through installing drivers for the home screen and the bootloader, and then you need to install one more yourself, the exact same way as the other two, but with the phone on the recovery screen (you can enter the recovery screen from the bootloader screen). You'll know it worked when using "adb devices" in command prompt lists the phone while the phone is on the recovery screen.
Hope that saves someone an hour or three.