Topological invariants for anomalous Floquet higher-order topological insulators