>>1097925a non-trivial solution is one where x is not equal to the zero vector, since the zero vector maps to zero for any linear transformation.
so you are looking for all vectors x that map to the zero vector. these solutions will form a subset of your input space known as the "nullspace" or "kernel".
it's easy to show that the nullspace is a subspace by taking linear combinations of elements in the nullspace (i can add two nullspace vectors that map to zero, and the result will also map to zero. same with multiplication).
a _very_ useful observation is that the output of a matrix is a linear combination of the columns. that means the columns span the possible outputs.
thus, you want to try to find ways to make a linear combination of the columns that sums to the zero vector. the wording of the problem gives you a good idea of how to do this.
for the second question, the m and the n refer the to number of dimensions in each space. R^n is an n-dimensional space over real numbers (meaning that a coordinate vector can be represented with n real numbers).
so the question is asking "if i take an m-D space and linearly map it to an n-D space, can that map be one-to-one, onto, both (bijection), or neither"