Our mechanistic understanding of fluid turbulence has substantially improved in recent decades due to the discovery of large numbers of unstable simple invariant solutions to the Navier-Stokes equations. Heteroclinic connections between these solutions have been hypothesised to play an important role in high-dissipation, intermittent `bursting’ events. However, standard methods of detecting simple invariant solutions are not suited to finding these connecting orbits, and consequently only a few have been discovered to date. Here, we introduce automatic differentiation (AD) as a robust technique for finding connections via gradient-based minimisation of a suitable loss function. We first use a new, fully differentiable point vortex solver [jax-pv] as a playground to test the loss-based approach. We demonstrate that AD can successfully find connections in non-integrable point vortex systems. We also discuss how AD can systematically explore the free energy landscape of a rotating superfluid (Campbell & Ziff, Phys. Rev. B 20, 1979). This discussion offers insight into experimental observations that show sequences of transient states before the global energy-minimising state is reached. We then extend our analysis to a two-dimensional Kolmogorov flow (monochromatically forced on the two-torus) using a fully differentiable Navier-Stokes solver (Kochkov et al, Proc. Nat. Acad. Sci. 118, 2021) to search for “bursting” connections between low dissipation relative equilibria.