The section with the burning buildings and the big guy in the middle of the court yard just past the shallow water?
If that’s the section you mean, then you can safely come back after progressing the main line a little more. Getting the axe and the beating the guy on the horse are good pre-requisites. That will align the difficulty to be more consistent with the main line path.