Performance issues with v24.0.2

Yeah we will soon cut the new release.
Update:
@rahst12 @Joschka @RJKeevil We have made a new release rc, v24.1.0-rc5. It has the cache changes as requested. Please give it a go and let us know what you guys think