To scale vertically or scale up, means to add resources to a single node in a system, such as adding memory or a faster hard-drive to a computer. To scale horizontally or scale out, means to add more nodes to a system, such as adding a new computer to a clustered software application.
It is often advised to focus system design on hardware scalability rather than on capacity. It is typically cheaper to add a new node to a system in order to achieve improved performance than to partake in performance tuning to improve the capacity that each node can handle.
See also: