I'd have to agree with dat838 if you only use BASIC physics - as basic as increase downforce, decrease final top speed as the drag coefficient will change.
Once you leave high school physics and start looking at things like grip coefficients of tyres, torque curves of cars, and a number of slightly more than basic physics considerations you should be able to demonstrate that corner entry and exit speeds will increase - increasing lap times - as I don't think converted every reaches the top speed of his car on the race track.
Have a read of something more than basic physics and then you can make some more informed comments.
By your rational when a F1 car loses it's front or rear wing it will actually go faster - and this is clearly not the case.
At least you qualified that according to very BASIC physics you opinion could be supported - gave me the clue where you could spend some time learning more about race car dynamics. Look into grip, downforce and weight transfer next.