Is your starter disconnected from the front starter motor brace?
# 25
http://www.tpocr.com/images/IPC/engelectipc.jpg
To remove the starter leave the harness attached for the moment, remove nut and washer ( # 24 & 25 ) then remove the two block mounting bolts # 27 slip the starter rearward to clear the front bracket # 25 and drop down the starter motor down just low enough so that you can remove the wires from the starter motor. Once the wires are off the starter is free to come down.
Note; sometimes you will have to loosen the front starter brace for clearance.