It’s super simple. Just include a push directive in your site definition. You can leave it as just that, and Caddy will use any Link HTTP headers to figure it out.

If you want more control, you can expand the directive and specify both the path and associated resources, like so: {
    root /var/www/example
    push / {

What this block does is say “for every request with a base of / (i.e. every request), Push the following 3 files.” You can customise the base path if you want to, and add more files if you need, but a block like the one above is what I’m using for this site.

You can find out full details in the Caddy Push documentation.