What causes terminal velocity is the balancing of forces, it is when the downward force of gravity is matched by the upward force of air resistance. When the forces acting on a moving object are balanced, that object will continue moving in a straight line at a constant speed.
As you correctly pointed out, if an object is fired downward, the force provided by air resistance would be greater than the force of gravity and the object would slow. But it the object slows the force of air resistance will decrease until the point where gravity and air resistance will be balanced. If the object is dropped then then the force of gravity is greater than the force of air resistance, the object will accelerate, the force of air resistance will increase until it balances the gravity.
Ok, now for an object falling through the earth.
If the object is dropped through the hole, the force of gravity is greater than the air resistance. The object accelerates. Eventually the gravity will decrease, the object will slow, but the air resistance will decrease as well. At no point does the air resistance overcome the momentum of the object. Gravity will continue to move the object downward (towards the centre). At the point the object reaches the centre, gravity is zero, but the object is still moving, it still has momentum. How fast the object is moving would depend on the shape and density of the object, but it would still have momentum. Because of that momentum it will travel past the centre point. At this point air resistance and gravity are acting in the same direction and the object slows, but it has passed the centre point.
The pendulum swings.