How to check if a varchar column is unique?

Question

Database design

There is a task:
1. Drive a list of words (surnames, first names, ranks, etc.) into 1 column
2. When adding a list, you need to check the uniqueness of the values, if not unique - do not write
3. Pull the list, but without duplicates back to a text file .
Let's deal with the first and last (import wizard). The question is how to ensure the uniqueness of values ​​in a column when creating a database (structure)? It is advisable not to use an index (no work with this database is required, and the volume of the database grows a lot) and not to use MERGE.
Now we have a variant with a primary key (without checking for uniqueness):

CREATE TABLE dic
( dic_id BIGINT NOT NULL IDENTITY(1, 1), /* Код слова */
dic_value VARCHAR(30) NOT NULL, /* слово */
CONSTRAINT PK_dic PRIMARY KEY (dic_id))
WITH (DATA_COMPRESSION = PAGE);
GO

Answer 1

it is necessary to separate the flies from the cutlets

you need to get uniqueness when loading?
Or when unloading?
in the first case, there was already a tip with a unique index. but as correctly noted, will discard duplicates with an error. and errors must be handled. here it seems like programming begins (and where is programming, your task can be solved in 1000 ways anyway) it
’s not very clear on the question whether there will be further work with the base? if work is not expected, and the DBMS is used only for filtering, then DISTINCT will help you. load everything as it is, and unload the query result from the database . it is possible to create VIEW . this is a more logical approach, especially if you break the data into columns - last names, first names, patronymics, titles, and so on. psSELECT DISTINCT ...
specifically for MS SQL and sensitivity to registers, dancing with tambourines is not needed, everything was invented before us (and for us) https://www.webucator.com/how-to/how-check-case-se...
not myself I've tried it too, I won't lie. but it seems possible to change for a separate table of the database, of course, if other applications are running on the server, and it is not desirable to change their behavior
https://docs.microsoft.com/ru-ru/sql/t-sql/stateme...
in case multi-user server, another backup option is to install a separate instance, you can even LocalDB or Express, and already set case-sensitive collation in it

Answer 2

What is the DBMS?
MySQL, for example, simply creates a unique index:

CREATE TABLE dic(
...
dic_value VARCHAR(30) NOT NULL UNIQUE
...);

Answer 3

This can only be done with a unique index. When adding a non-unique record, it will throw an error.
Otherwise, you will have to manually do the same thing as him, much less efficient in terms of speed and resource costs.
ps if there are a lot of columns, then instead of adding them to the index, you can add only a hash row based on them (i.e. add another field in the table and when adding, manually programmatically or using sql, the hash from the sum of concatenate your columns, only between rows add some character to separate the columns), just take something not the simplest so that the probability of collisions is as small as possible (collision in your case is a false positive on lines that are not duplicates)