In case of useEffect() - you're right. A callback returned from the function passed to it would be used for cleanup.
However, the provided examples are just custom hooks - functions that composite hooks to build more complex functionalities. In this case, nothing special is done with the returned callback - it can simply be retrieved when the custom hooked is used and called from there.
E.g. in the custom useTick() hook, the returned callback stops the running interval. Thus, after the hook is used in the component and callback is retrieved, it's then passed to onClick, so that the ticking will stop after clicking the button.
I hope it's clear now.