If I have a Chapter, do I have to create a new Overview Map to change it into a Section?

No. 

You only need to change the Chapter Title into a Section Title via "Styles" in the FS Pro for Word ribbon.

You can proceed like this because an overview map has always the same structure:

• Title
• Block "Introduction"
• Block "Contents"