Coat grill grates with the cooking spray. Set grill to medium-high. Bring a large pot of salted water to a boil. Add the potatoes to pot and cook 7-10 minutes, until just tender. Drain in a colander and pat dry with paper towels. Cut potatoes in half. Toss potatoes with 2 tablespoons oil and season with salt and pepper.
Add potatoes to grill cut-side down. Cook potatoes 5-10 minutes, until browned and tender, turning occasionally.
Meanwhile, in a large bowl, combine remaining 3 tablespoons oil, vinegar and mustard. Season with salt and pepper. Stir in the celery, herbs and potatoes. Serve immediately.
Tip: If your potatoes are smaller and might fall through the grill grates, cook the potatoes in a grill pan on the stovetop for the same amount of time.