made a simple code :
https://www.babylonjs-playground.com/indexStable.html#ILJXTY#6
Why camera gravity is not applied if camera don't move?
Press "J", camera gravity will not applied, if you don't move after you can wait it will never apply without moving cam, when you moving cam with Z camera gravity is just applied for a moment if you up z its the same problem.