فَلَمَّا رَأَوْا بَأْسَنَا ...

So when they saw Our punishment,

means, when they saw with their own eyes the punishment which came upon them, they said,

... قَالُوا آمَنَّا بِاللَّهِ وَحْدَهُ وَكَفَرْنَا بِمَا كُنَّا بِهِ مُشْرِكِينَ ﴿٨٤﴾

We believe in Allah Alone and reject (all) that we used to associate with Him as (His) partners.

means, they affirmed that Allah is One and denied the false gods, but this was at the time when excuses were to no avail.

This is like what Fir`awn said as he was drowning:

ءَامَنتُ أَنَّهُ لا إِلِـهَ إِلاَّ الَّذِى ءَامَنَتْ بِهِ بَنواْ إِسْرَءِيلَ وَأَنَاْ مِنَ الْمُسْلِمِينَ

I believe that none has the right to be worshipped but He in Whom the Children of Israel believe, and I am one of the Muslims. (10:90)

But Allah said:

ءَالَنَ وَقَدْ عَصَيْتَ قَبْلُ وَكُنتَ مِنَ الْمُفْسِدِينَ

Now (you believe) while you refused to believe before and you were one of the the corrupters. (10:91)

meaning, Allah did not accept this from him, because He had answered the prayer of His Prophet Musa, when he said,

وَاشْدُدْ عَلَى قُلُوبِهِمْ فَلاَ يُؤْمِنُواْ حَتَّى يَرَوُاْ الْعَذَابَ الاٌّلِيمَ

And harden their hearts, so that they will not believe until they see the painful torment. (10:88)