This is a true story -- I was walking in downtown St. Petersburg toward The Pier with a student while we were having a lesson. A woman with a Canon DSLR and large camera bag walked past us. Ten seconds later we heard a crash and a scream. The woman had not closed her camera bag all the way and some of her gear fell onto the sidewalk. Even before this I was already obsessed with closing my camera bag immediately after taking out what I needed, even if I knew I would be going right back into my bag only a minute later. It is a pain in the neck no doubt to zip and unzip and zip and unzip all the time, but that minor annoyance is nothing compared to the ill feeling that woman with the spilled gear might have felt.
Even if your camera bag is not in motion while left open, I still think it's dangerous. In the photo above my camera bag is set in a chair. I made sure it was firmly in the chair. However, I have set countless things firmly in chairs, on countertops, totally sure there is no way those things could fall over, yet many have. At least if my bag falls out of the chair closed, the ample, stiff padding of my Lowepro bag will protect it.
What if you see something interesting and you just grab your bag and head towards it? If it's open you will maybe risk spilling stuff just grabbing it, or at the least you'll have to take a few seconds to close it up then go, which you may know in photography, a few seconds can make all the difference between getting a shot and getting nothing.
Therefore, I recommend becoming obsessive about closing your camera bag right back up.