On the Home settings screen, click Allow Speaker & TV Access, then choose an option.

You can allow everyone, anyone on the same network, or only people you invite to share the home. You can also require a password that allows speaker access. To learn more about HomePod speaker and Apple TV access, see the Apple Support article: Share control of your home