The flowers are absolutely beautiful—definitely worthy of being called a world-class spot. I recommend bringing your own food though, since there aren't many options for sale inside the park.
If you want to really take in all that Toshogu Shrine has to offer, half a day just isn't enough. I suggest staying overnight in Nikko so you can also explore places like Lake Chuzenji nearby.