Colonel Abdirashid Hassan Hashi, known as “Gabi Gabi”, was shot dead outside a mosque in Galkayo, where he performed evening prayer.
The gunmen escaped from the area after the attack, according to sources.
No group has claimed the killings but the security officials say Al-Shabab militant are suspected to have carried out the attack.
In the past few months, top Puntland officials were killed in Galkayo in Mudug. Puntland lawmaker Abdihakim Moalin Dahir was killed in Galkayo by armed men with pistols in late May.