Hallgrimsson, who replaced Stephen Kenny as manager, was happy with the performance but said there was room for improvement.
“We can be happy with a lot of things in the performance. I thought in the first half, there was one mistake and we were punished,” he added.
“I thought we played pretty good at the end of the first half and we kind of built on that in the second half.
“It wasn’t a perfect match, they had two chances and we could have been punished again.”
After leaking goals in their opening two matches, Hallgrimsson was pleased with how much his side restricted Finland, who were also without a win heading into the match in Helsinki.
“We didn’t give them a lot of goalscoring chances. We gave it to them, it’s not like they played through us.
“Overall, I think we should be happy with the performance.”
The former Iceland manager also said he was pleased with how Collins, who was handed the armband for the match, responded after his error.
“You cannot do anything about mistakes, that is something that happens.
“We cannot criticise anyone for making a mistake, but we were pretty happy with what we were doing, especially at the end of the first half.”
The Republic of Ireland are next in action on Sunday away to Greece, who stunned England to maintain their perfect Nations League record.
“It is a difficult game in three days,” warned Hallgrimsson.
“There were some tired legs on the pitch. We need to start thinking about Greece.”