Fewer prime-age American men are in the labor force than in many other advanced countries, a possible downside of a flexible U.S. job market.