The hostname is the actual name of the server where MySQL is installed. If everything is on the one server (i.e. it's not a 'complicated' cluster), then the hostname is simply "localhost"
So yes, it's *usually* localhost.
If (say) it's a cluster of servers though, then the DB hostname would be the IP address of the server where MySQL is installed on.