Is it safe to use my multimeter with a 12 Volt battery?

yes. the battery is DC, so put the meter on 10A DC in series with the connection the the inverter, and scale down until you get an appropriate reading.