I got on the subject of exercise with a man who was in less-than-peak physical condition. I asked him when was last time he had gone all-out physically. He was 65 years old. He said to me, "John, it's been decades".
Your body adapts positively to the stress of exercise. If you do not stress the body the body's capacity to handle the stress of exercise diminishes over time.
Your fitness, your health, and your ability to ward off sickness and injury diminish. Your body down-regulates its ability to handle sugar, your metabolism declines, and the loss of flexibility and strength increase the likelihood of injury. With that injury – a bad back, frozen shoulder, fragile knees, you name it – comes even more inactivity and a still lower level of fitness and health.