Skip to main content

Implementing the research design

Chapters 10, 13 & 23

Doing your research and analysing your results

  • Keep good records of what you do, e.g.:
    • if doing a survey, keep good records of who has replied, so that you know who should be sent reminders. If you use software like Survey Monkey this can be easily tracked for you.
    • if doing participant observation, remember to keep good field notes and not to rely on your memory
  • Make sure that you are thoroughly familiar with any hardware you are using in collecting your data, such as tape recorders for interviewing (including recorders on your phone!), and make sure it is in good working order
  • Do not wait until all your data have been collected to begin coding
  • Remember that the transcription of tapes with recorded interviews takes a long time, in the region of 4-6 hours per one hour of recorded conversation
  • Become familiar with any data analysis packages as soon as possible
  • See Quantitative and Qualitative data analysis

Exercise: structured interviewing

A manufacturer of farm implements wished to find out why the response to a first design of animal drawn plough, specifically designed as a cheap but effective implement for developing country farmers, was rather poor. A combination of observation and exploratory research suggested the following factors played a role in shaping the negative attitude of farmers to the design:

- Does not ridge

- Does not work for inter-cropping

- Far too expensive

- New technology too risky

- Too difficult to carry

The company wanted to know which factors were the most important. If it solved these then the other factors may be of minor significance in preventing widespread adoption of the plough. The alternatives were to abandon the product's re-development or redesign it completely, in which case it may be expensive and face even further objections. The company decided to sample 200 farmers who used the plough.

1
Question

Use Microsoft Word or other suitable software to design a 'paired comparison' scale/test to enable the company to rank the most to least important farmer objections. How many pairs are needed?

With five factors as objections, each of the objections need to be paired by the researcher to compare them. This would require 10 pairs, calculated as follows:

Equation

The questions might be put as follows:

Which of the following was the more important in making your decide not to buy the plough?

 

The plough was too expensive

  

It proved too difficult to transport

 


In most cases the question, and the alternatives, would be put to the farmer verbally. The farmer then would indicate which of the two is the more important and the researcher would tick the appropriate box. The question would then be repeated with a second set of factors and the process repeated until all the possible combinations were exhausted i.e. 10 pairs. It would be good practice to mix the pairs to avoid systematic bias. The researcher would try to ensure that any particular factor would be sometimes the first of the pair to be mentioned and sometimes the second. The researcher would never have one factor systematically compared with the others in succession as bias would ensue.

Check your answer

2
Question

Use Microsoft Word or other suitable software to design an appropriate simple data analysis technique to enable the company to find the most and least important objections of the farmers. Making any assumptions you wish regarding the number of farmers responses to the paired comparisons, show how analysed data can be used to enable the company to make a decision on what to do.

A simple data analysis technique would be to label the factors and record the farmer responses to the paired comparisons in a matrix. For example:

Label the factors A-E as follows:

A = does not ridge

B = far too expensive

C = new technology too risky

D = does not work for inter-cropping

E = too difficult to carry

Assume all the 200 farmers responded. The data is then arranged in a matrix. Further, assume that the data is arranged that we read side to side. This means, for example, that 164 out of 200 farmers said that the fact that the plough was too expensive was a greater deterrent than the fact that it was not capable of ridging. Similarly, 174 farmers said that the plough's inability to inter-crop was more important than the inability to ridge when deciding to buy the plough.

Preferential matrix

A B C D E
A 100 164 120 174 180
B 36 100 160 176 166
C 80 40 100 168 124
D 26 24 32 100 102
E 20 34 76 98 100


From the grid, the rank order of factors is:

Most important E = too difficult to carry
D = does not inter-crop
C = new technology too risky
B = too expensive
Least important A = does not ridge


From this, it is more important for the company to concentrate on improving transportability and, if possible, to give it an inter-cropping ability, rather than focusing on its ridging capabilities. This may be a cheaper alternative than undertaking a complete redesign.

Check your answer

Exercise: structured observation

A university is wishing to revalidate its Computing Masters Degree programmes, which had become rather 'tired' as a result of rapidly changing computing and allied technology. Falling enrolment numbers and student/employer feedback tended to reinforce the view that the current programmes had run their course. As part of the development process, the university decided to find out what programmes its competitors offered. This would help it to identify any gaps in the product/market provision which could be potentially plugged with a new programme. In addition, such information would help the university with its marketing effort. The university felt that, besides the programme content, other important but less 'physical' aspects of the university study environment, might also be contributing to falling numbers, for example, a 'research culture'.

The university identified four benchmark universities as its target competitors. Its task now is to:

- Identify the 'physical' competitive aspects of current and potential new programmes, for example, title, and price for the programme

- Identify the 'subjective' aspects of the programme, for example, 'research culture'

- Use the information to help develop new Computing programmes and market them.

3
Question

Making any reasonable assumptions you wish, design a structured observation method (using, for instance, Microsoft Word) to assist the university achieve its stated task.

This question could be answered in a number of ways. For example, the university could merely ask the four benchmark universities to send the prospectuses and programme fees. It could do this in a number of ways. It may also pick up the same from exhibitions, open days, or student recruitment fairs. However, this is likely to result in limited detail as its rivals would probably not readily reveal any special student fee discounts or specific detail of the programmes. This would certainly not reveal the more 'subjective' aspects of the programme, although a trawl through the Times Higher Education supplement might give insight into, say, 'research culture'. However, this is unlikely to be specific enough.

A viable alternative would be to adopt a 'mystery shopper' method and use a structured observation technique. The university may send a researcher (mature PhD student?) to its four competitors on Open days, armed with a structured observation form on which he/she records the desired data. The observations could be coded 1 to 5, 1 = very poor to 5 = very good, to record the degree of difference between the five universities. For example, if one university had a computing department where all staff were research active as defined by the Quality Assurance Agency, then it may be coded a '5' on that activity. One with 75% of its staff research active may get a coded score of '4', and so on. The steps in the research process would be as follows:

  1. The university would have to decide on its research objectives, and these are more or less the three elements of the task as stated.
  2. It needs to decide on who would carry out the research, when, and how. If it is to be a mystery 'shopper method', then an observation form would need to be designed and coded.
  3. The university would need to decide which 'objective' and 'subjective' aspects it would want to find out. Examples of these may be:

'Objective' elements:

      • Programme titles
      • Programme contents (detailed)
      • Mode of programme delivery (full time, part time, e-learning etc.)
      • Dates programmes run and duration
      • Programme fees
      • Any special discounts
      • Programme staff
      • Accommodation for both teaching and living
      • Learning resources and support including computing facilities, hard ware and software, library facilities
      • Employability prospects after programme study
      • Student Union facilities
      • Etc

'Subjective' elements:

  • Research culture
  • Study ambience
  • Reputation of institution
  • Reputation of staff
  • Friendliness of staff
  • Student 'care'
  • Value for money of programme
  • Safety and security of location
  1. Put these into a structured observation form, carry out the observation and record he data on each University, for example:

Observation form

Aspect Own Uni Uni 1 Uni 2 Uni 3 Uni 4
Objective:
Title Computing Information systems Computing Networks Computing Computing and Information Systems
Content x, y, z, x, a, b x, y, a x, y, z, a, b, x, y, z, a, b.
Mode of delivery Ft, Pt and e-learning Ft and Pt Ft and e-learning Ft Ft and Pt
Duration I year 1 year 1 year 1 year 1 year
etc
Subjective:
Research Culture 1 5 3 2 1
Study ambience 4 3 2 2 5
Institution reputation 2 4 5 2 1
Staff reputation 1 1 3 5 4


NB. For Subjective elements code:

Ft = Full time, Pt = Part time

1= very poor, 2 = poor, 3 = non-existent, 4 = good, 5 = very good

  1. Interpret the results, draw conclusions (comparing itself with its competitors on the 'objective' and 'subjective' aspects).
  1. Make recommendations on the titles, number and type of programmes, curriculum design, pedagogy, fees and marketing aspects based on the conclusions.
Check your answer

Exercise: documents as sources of data

Take a book of old photographs of your city/town through the ages or annual reports from the last ten years of a public sector organization (commercial or non-profit organization).

From the portfolio of photographs answer the following:

4
Question

What social/economic changes in the city/town are displayed in the photos throughout the ages?

Changes should be noted in :

- Economics - e.g. type, value and quantity of vehicles, style and type of clothes, manufacturing versus service industry

- Socio-cultural - e.g. age of people, number walking the streets, type of dress, age distribution mix, families versus singletons, type of shop, communication signage. Pedestrianization, 'suits' versus 'overalls'

- Legal - e.g. number of street regulatory signs

- Technology - e.g. mp3 players, Ipods, electrical goods shops, cars versus trolley buses/trams

Check your answer

5
Question

What cultural context/meaning can you get from the photographs that could be used to advantage in current day advertising and marketing campaigns?

Look for a theme, occasion, or scene which typically describes the 'culture of them for example an 'essentially Georgian' scene, or a 'family values orientation' and so on. An old city square or a day out at the seaside respectively may reflect 'essential cultural values' associated with the time. Evaluate whether these could be transformed into the present day and promoted to advantage. The 'family values orientation' could be reflected in a 'return to family values,' campaign, for example, for advertising a family day out in a zoological park.Check your answer

From the annual reports answer the following:

6
Question

Has the terminology to describe 'customers' changed and become more inclusive over the years?

A qualitative content analysis could be performed on the documents:

Look for changes in the term itself from 'consumer' to 'customers' and try to explain why

Look for changes in the scope of the term from consumer (or customer) to 'stakeholder' and who are included in the term, e.g. customers, suppliers, distributors, governments, pressure groups, shareholders, etc. Note the time the change occurred and evaluate what lead to this change.

Check your answer

7
Question

Has the company's attention to environmental matters and social responsibility been given more attention over the years? How?

Again, trace the history of events, the broadening of the scope of the terms and what is included in them.

Try to evaluate why the company has broadened its attention to these matters. See if the historical change correlated to changes in legislation, indirect or direct actions of pressure groups or a crisis in the organization itself.

Identify how the organization has given more attention to these matters, what specific activities have been put in place, who has been made responsible for them, and how important is the attention given to them by identifying how high in the organization are the responsible persons.

Check your answer