Control-click the folder, choose Delete Folder, then click Delete.
The folder and the albums within it are removed from the library and from iCloud, but the items in the albums remain in your photo library. If you accidentally remove a folder, choose Edit > Undo Delete Folder to restore it.
To remove an album from a folder, click the folder in the sidebar, then Control-click the album on the right and choose Move Album Out Of [folder name].