With the increasing need to apply modern software techniques to hardware design, Verilog, the most popular Hardware Description Language (HDL), plays an infrastructure role. However, Verilog has several semantic pitfalls that often confuse software and hardware developers. Although prior research on formal semantics for Verilog exists, it is not comprehensive and has not fully addressed these issues. In this work, we present a novel scheme inspired by previous work on defining core languages for software languages like JavaScript and Python. Specifically, we define the formal semantics of Verilog using a core language called $\lambda_V$, which captures the essence of Verilog using as few language structures as possible. $\lambda_V$ not only covers the most complete set of language features to date, but also addresses the aforementioned pitfalls. We implemented $\lambda_V$ with about 27,000 lines of Java code, and comprehensively tested its totality and conformance with Verilog. As a reliable reference semantics, $\lambda_V$ can detect semantic bugs in real-world Verilog simulators and expose ambiguities in Verilog's standard specification. Moreover, as a useful core language, $\lambda_V$ has the potential to facilitate the development of tools such as a state-space explorer and a concolic execution tool for Verilog.