We propose a fast local level set method for the inverse problem of gravimetry. The theoretical foundation for our approach is based on the following uniqueness result: if an open set D is star-shaped or x3-convex with respect to its center of gravity, then its exterior potential uniquely determines the open set D. To achieve this purpose constructively, the first challenge is how to parametrize this open set D as its boundary may have a variety of possible shapes. To describe those different shapes we propose to use a level-set function to parametrize the unknown boundary of this open set. The second challenge is how to deal with the issue of partial data as gravimetric measurements are only made on a part of a given reference domain Ω. To overcome this difficulty we propose a linear numerical continuation approach based on the single layer representation to find potentials on the boundary of some artificial domain containing the unknown set D. The third challenge is how to speed up the level set inversion process. Based on some features of the underlying inverse gravimetry problem such as the potential density being constant inside the unknown domain, we propose a novel numerical approach which is able to take advantage of these features so that the computational speed is accelerated by an order of magnitude. We carry out numerical experiments for both two- and three-dimensional cases to demonstrate the effectiveness of the new algorithm.