First things first: bikes are not allowed on any of the trails in the refuge, only the gravel roads that are open to vehicles. You probably won't see any cars on these roads so it's a great place to get a long training ride in on the dirt and rocks. In many places the gravel is deep and loose but there are stream crossings and good scenery to make up for the crummy surface conditions. I saw wild turkeys and deer on my ride.
From Gray, GA take GA Hwy 11 north/west toward the Round Oak community. Turn left onto Roud Oak Juliette Rd. and look for signs for the visitors center. Park at the center or, as I did, at the small church about 2-3 miles past the visitor's center on the right.
http://www.singletracks.com/item.php?c=1&i=3266