You may have heard buzz in the business world about text analysis, artificial intelligence, or big data. But if you’re skilling up to advance your career or to solve an existing company problem, it is not obvious why text analysis matters. Many businesses operate without data science insights from text analysis (and we all know a few businesses likely to continue for decades more), but there is no doubt that interest in text analysis for business insights is growing.
Data literacy is at the heart of research
Twenty years ago, the data available for business insights could be handled by a few specialized individuals. Today, it is clear that digital data is the main record, not just for businesses, but for all of human society. Even small businesses often have more data than they have the expertise to understand and put to use. The ability to search and manipulate digital records--to clean them and to connect them--has become essential for decision-making from the C-suite to middle management. The vast majority of data in any business is unstructured, text data including internal email, user feedback, social media, and transaction data. This firehose of data quickly becomes overwhelming. That’s why data skills are in demand.
Data skills are in demand
Does anyone know the best job according to Glassdoor in 2016, 2017, 2018, and 2019? One reason data scientists are in-demand because they can use text analysis to create business insights. They might use it to extract the sentiment from customer service calls, to gather customer demographic or location data from social media, or to redact sensitive information from research. Text analysis is behind the auto-suggest on your phone, it filters the spam out of your email, it helps suggest movies for you on Netflix. In this information age, with the onset of “big data,” text analysis helps us “read” and “interpret” more than is humanly possible.
For businesses that have significant data, the primary advantage that text mining offers is an ability to drive (and in some cases, automate) key business decisions based on data insights from large-scale, unstructured textual data. Without a doubt, numerical data remains the most compelling and tractable form of business data, but many businesses also have a significant body of textual data that sits dormant, in part, because no one on staff possesses the skills to unlock its potential.
If your business is of a certain size, you’re certainly gathering data on your customers and users. Maybe you have some text data that is structured (or data you wish was structured) in the form of customer names, addresses, email addresses, or past purchases. Text analysis can help turn unstructured data into structured data, but it can also help make unstructured textual data tractable and actionable. Think about the data your company has.
- Do you collect written feedback from customers?
- Do you offer customer service or call-center help? “This call may be monitored or recorded for quality assurance purposes”
- Do your products have written documentation?
- Does your marketing team interact with social media?
- Does your business use email? In 2019, 188 million emails were sent every minute.
For many, text analysis sounds *potentially* powerful and useful, but the reality remains that learning text analysis is not a trivial task. To become proficient, you’ll have to learn a programming language like Python or R. If you’re not doing formal coursework in linguistics or computer science, it may feel like getting started is impossible. That’s as true for students as it is for professors and librarians. The good news is that programming and text analysis, like any skills, can be learned to a greater or lesser degree. Mastering a language can take many years, but you don’t need to master programming to get started.
If you want to do international business, it is very helpful to speak a second language. Still, there are plenty of successful business people that never learn a second language (or learn just enough to navigate local resources significant to their business interests). You don’t need to know everything about Python or everything about text analysis. You just need to learn enough to leverage the data your company has available.
How long will it take me to learn text analysis?
The answer to, “How long will it take me to learn text analysis?” depends on the skills you need to learn. The problem for many people is the possible applications for text analysis are not clear, so they are not in a good position to decide what to learn (and how much). If you have no programming experience, it is reasonable to expect you could be doing beginner-level text analysis methods in a few weeks and intermediate-level text analysis methods in a month. But what methods?
Let’s consider a few of the current methods. What are they? Why would you use them? How long will it take to apply them?