This was in the news last year when many users reported that disabling app usage access for Google’s “Digital Wellbeing” feature noticeably improved their phone’s performance.
While I don’t have Digital Wellbeing on my phone (low end Lenovo P2 running LineageOS 16 and pico GApps), I did find a few other apps using this permission. And would you know, blocking them (particularly Google Play Services and Google Play Store) made my phone noticeably faster indeed.
On my phone, with near-stock Android 9 UI, this option is available at Settings > Apps & notifications >Advanced > Special app access > Usage access. Depending on your phone’s Android version and customization, you may find it elsewhere.
Note that usage access is necessary for some apps to function properly, so be prepared for a little trial and error.